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CHAPTER  I 
INTRODUCTION 

The  Probleia 

"It  is  becoming  increasingly  obvious  that 
the  full  potential  of  information  processing 
will  not  and  cannot  be  realized  until  computers 
and  their  applications  can  be  managed  reliably 
in  an  economic  sense  (Weinwurm,  1968,  p.  329)." 

Weinwurm's  above  comment  at  the  1968  National  Conference 
of  the  Association  for  Computing  Machinery  could  well  be  taken 
as  the  underlying  imperative  for  this  research.   Computers  have 
had  great  impact  on  our  economy,  on  organizations,  and  on 
individuals  in  the  past  fifteen  years.   They  have  been  cursed 
and  praised.   The  literature  of  virtually  every  discipline 
contains  increasing  references  to  computers. 

However,  one  recurrent  theme  that  keeps  appearing  over 

and  over  again  when  computerized  information  systems  are  under 

discussion  is  the  failure  of  organizational  managements  to 

capitalize  on  the  potential  offered  by  computer  technology. 

Many  accounts  of  such  management  shortcomings  could  be  cited, 

and  some  will  be  cited  further  on,  but  Reynolds  (1968)  summed 

up  the  essence  of  all  of  them  quite  succinctly  when  he  said: 

"The  common  complaint  among  people  who  must 
plan  for  and  manage  the  development  of  computer 
program  systems  is  that  the  products  are  almost 
always  finished  over  budget  and  late,  and  they 
hardly  ever  do  what  they  were  intended  to  do  (p.  334)." 
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John  Diebold  (1969),  a  pioneer  in  the  information 
systems  field,  has  predicted  that  "by  1985,  the  computer  will 
have  become  central  to  the  nervous  system  of  the  corporation," 
Diebold  (1969)  has  also  pointed  out  that  the  "share  of  personnel 
and  software  cost  in  the  total  ADP  mix"  has  substantially 
increased  in  the  past  five  years  to  the  point  where  these  costs 
often  amount  to  twice  the  annual  hardware  cost.   Both  of  these 
factors  lead  one  to  the  conclusion  that  the  management  of 
management  information  systems  (MIS)  should  be  the  focus  of 
much  attention  and  research. 

The  Need 

Before  proceeding  to  a  statement  of  the  purpose  for 

the  present  research,  the  following  rather  extensive  quotation 

from  Weinwurm  (1968)  emphasizes  the  need  for  the  type  of  research 

conducted  by  the  author  and  reported  in  this  thesis: 

"Of  the  explosive  and  unremitting  change  that  is 
characteristic  of  information  processing  there 
can  be  no  doubt.   Nonetheless,  there  is  a  great 
difference  between  noting  the  pace  and  dynamism 
of  the  field  and  asserting  that,  as  a  result, 
management  research  is  bound  to  be  wasteful-- 
no  matter  what  the  investment  in  it,  regardless 
of  the  probability  of  success,  and  whatever  the 
savings  that  may  result.   The  empirical  evidence 
that  can  be  marshalled  in  support  of  such  a 
compound  assertion  is  rather  meagre.   Indeed,  it 
is  doubtful  whether  a  deliberate  policy  of 
investing  in  management  research  on  a  scale  that 
is  in  some  substantive  sense  proportional  to  the 
magnitude  of  technical  development  has  ever  been 
seriously  attempted. 
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"Simllarly  since  all  possible  innovations  and 
contingencies  obviously  cannot  be  anticipated, 
learning  by  experience  is  a  necessary  concomi- 
tant to  any  technological  advance.   Adopting 
'trial  and  error'  as  a  governing  policy  is 
another  matter  altogether.   There  is  consider- 
able reason  to  believe  that  the  real  costs  of 
information  processing  'trials'  are  far,  far 
higher  than  is  generally  recognized.   In  fact, 
due  to  the  ingenious  ways  by  means  of  which  we 
have  learned  to  obfuscate  learning  inefficiencies, 
the  real  costs  are  very  likely  indeterminate. 
In  any  case,  no  serious  attempt  has  ever  been 
made  to  demonstrate  that  'trial  and  error'  is 
the  most  economically  effective— or  the  most 
expeditious— way  of  advancing  the  state  of  the 
management  art;  perhaps  because  such  a  demon- 
stration may  well  involve  contradictions  of  a 
rather  fundamental  nature. 

"Finally,  there  are  undoubtedly  rather  basic 
similarities  between  information  processing  and 
any  other  process  that  involves  men  and  machines. 
At  the  same  time,  these  resemblances  can  be 
carried  beyond  the  point  of  supportability. 
While  the  general  validity  of  certain  principles 
of  good  and  effective  management  can  be  assumed, 
information  processing  in  many  respects  differs 
from  the  manufacturing  processes  upon  which  much 
of  our  management  knowledge  is  based.   To  the 
extent  of  these  differences --which  are  expecially 
evident,  for  instance,  in  computer  programming— 
literal  and  indiscriminate  application  of  tradi- 
tional management  principles  can  on  occasion  evoke 
rather  preposterous  management  policies.   Hence, 
the  critical  issue  is  not  whether  well-established 
management  methods  are  entirely  relevant  or 
whether  completely  new  variations  must  be  developed, 
but  which  combination  of  the  old  and  the  new  is 
most  appropriate  to  and  effective  under  each 
circumstance. 

"In  any  case,  whever  the  'truth' ,  collectively, 
lies,  one  would  expect  these  differences  of 
opinion  to  be  discussed  by  the  leading  experts, 
with  the  support  of  a  reasonable  assortment  of 
facts  Instead  of  assertions,  in  and  through  the 
formal  meetings  and  journals  that  the  relevant 
societies  provide  for  such  purposes.   That, 
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after  all,  is  the  professional  and  scientific 
way  such  issues  are  resolved.   Regrettably, 
very  little  if  anything  of  this  kind  has 
happened.   It  is  true  that  there  have  been 
hundreds,  and  probably  thousands,  of  discussions, 
symposia,  articles,  tutorials  and  the  like, 
during  the  past  10  years,  but  nearly  all  have 
been  on  a  transparently  superficial,  often 
amateurish  and  promotional  level.   And  one 
finds  it  difficult  to  hear  of  comprehensive, 
comparable,  or  reliably-analyzed  experience- 
da  ta — let  alone  the  management  measures  and 
standards  that  derive  from  such  data— anywhere. 

"I  believe  that  we  cannot  abide  this  situation 
any  longer.   It  is  high  time  that  the  management 
of  information  processing  was  brought  into  the 
mainstream  of  professional  discussion;  i.e., 
that  theses  be  formally  advanced  and  supported 
by  facts  in  a  respectable  manner;  and  that 
this  be  done  'on  the  record'  so  that  others 
will  be  stimulated  to  counter  in  the  same 
fashion." 


"While  the  lack  of  a  generally  available, 
accepted,  or  applicable  management  methodology 
is  a  serious  problem  in  nearly  every  area  of 
information  processing,  I  believe  most 
experts  would  agree  that  the  greatest  con- 
fusion and  the  most  critical  need  is  in  the 
management  of  computer  programming --in  the 
broad  sense  of  the  term.   Horeover,  software- 
related  problems— by  every  present  indication- 
will  increase  far  more  rapidly  in  their  scope, 
complexity  and  economic  impact  than  will  their 
hardware  counterparts  during  the  forseeable 
future  (pp.  330-331)." 


Purpose 

The  purpose  of  the  research  detailed  in  this  thesis 

can  perhaps  best  be  summarised  by  the  question: 

What  organizational  and  procedural  factors  are 
correlates  of  success  with  MIS  projects? 
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Raising  such  a  question  and  pursuing  its  answers  are  predicated 
on  the  hypothesis  that  there  are  certain  organizational  and 
procedural  variables  which  have  a  meaningful  influence  on  the 
success  or  failure  of  a  given  MIS  project,  and  that  these 
variables  can  be  identified.   Further,  upon  identification, 
such  variables  can  be  manipulated  in  certain  ways  under  certain 
conditions  to  enhance  the  probability  of  success  with  a  given 
MIS  project. 

Definition  of  MIS 

Before  going  further,  it  is  necessary  to  define  just 
what  is  meant  by  a  "management  information  system"  since  there 
does  not  appear  to  be  much  consensus  on  a  definition  among 
either  authors  or  practitioners  in  the  field  (Dickson,  1970). 
One  can  arbitrarily  designate  any  system  or  method  that  provides 
information  for  the  decision-making  functions  of  management  as 
a  MIS.   A  great  many  sources  of  management  information  beyond 
the  scope  of  the  usage  in  this  thesis  would  thus  qualify  as  a 
MIS.   (For  example,  the  Wall  Street  Journal.)   However,  the 
term  MIS  has  only  come  to  be  used  extensively  since  the  advent 
of  electronic  computers  for  processing  Information  to  aid 
managerial  dec is  ion -making.   In  fact,  it  would  appear  that  a 
great  many  people  equate  MIS  and  the  computer. 

It  is  this  latter  equation  that  the  author  considers 
to  be  incorrect.   There  have  been,  and  are  today,  many  kinds  of 
computer  applications  which  should  be  classified  as  data  processing 
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applications  as  opposed  to  MIS  applications.   The  perennial 
favorite,  when  such  a  distinction  is  raised,  is  a  payroll 
application  which  is  merely  converted  from  one  form  of  process- 
ing (manual  or  punched  card)  to  some  other  form  (computer). 
Another  example  of  a  non-MIS  computer  application  would  be  an 
inventory-control  system  which  merely  does  on  the  computer  what 
stock  clerks  were  doing  with  card  files,  and  nothing  more. 
However,  the  inventory-control  application  could  become  a  MIS 
if  upgraded  to  the  point  that  summarized  or  analyzed  data  were 
made  available  to  management  for  the  express  purpose  of  making 
decisions  on  such  questions  as  balancing  workloads,  make  or 
buy,  and  optimal  forms  of  distribution  to  minimize  cost  and 
maximize  service.   The  last  example  might  be  viewed  as  the 
threshold  for  qualifying  as  a  MIS  application,  since  some  of 
the  kinds  of  decisions  cited  could  be  programmed  and  handled 
by  the  computer  under  certain  circumstances.   The  important 
distinction,  then,  is  that  for  a  computer  application  to  be 
considered  a  MIS  application,  a  manager  at  some  level  (not  just 
the  corporate  executives)  must  be  receiving  information  from 
the  computer  which  aids  him  in  making  decisions  within  his 
domain  of  responsibility.   Such  decisions  could  be  of  either 
a  planning  nature  or  a  control  nature,  or  both. 

The  Focus 
Three  additional  points  require  elaboration  with 
respect  to  the  stated  purpose  of  this  research.   First,  the 
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focus  of  the  research  reported  here,  or  "element",  using  the 
terminology  of  Lazarsfeld  and  Menzel  (1961),  is  the  MIS  project. 
This  is  a  distinct  departure  from  previous  empirical  studies 
of  various  aspects  of  computer  usage  in  organizations.   Generally, 
the  element  studied  has  been  the  overall  organization  or  some 
segment  of  the  organization.   For  example,  there  has  been 
considerable  attention  devoted  to  the  impact  of  computers  on 
the  organization  (Shultz  and  Whisler,  1960;  Myers,  1967;  Reif, 
1968).   Another  approach  used  in  studying  the  organizational 
implications  of  computers  has  been  to  use  the  information  systems 
function,  or  "computer  complex",  as  the  element  (Reichenbach  & 
Tasso,  1968).   This  latter  method  involves  trying  to  evaluate 
the  impact  of  various  organizational  variables  on  the  effective- 
ness of  the  overall  computer  effort. 

The  assumption  here  is  that  if  methods  can  be  found 
to  increase  the  probability  of  success  with  specific  MIS  projects, 
the  overall  success  of  the  total  MIS  effort  will  logically  follow. 
While  this  is  a  strong  assumption,  it  is  further  assumed  that 
an  organization  doing  a  good  job  on  a  project  by  project  basis 
will  not  completely  neglect  long-range  MIS  planning  and  integra- 
tion.  At  any  rate,  it  is  believed  that  more  meaningful  hypotheses 
can  be  posed  and  tested  for  the  individual  MIS  project  than  for 
the  entire  MIS  effort  of  an  organization.   Further,  there  is 
need  to  evaluate  the  effectiveness  of  given  practices  in  more 
discrete  situational  contexts  than  the  total  organization, 
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since  project  characteristics  may  possibly  vary  in  such  a  way 
as  to  call  for  differential  treatments. 

The  second  point  requiring  elaboration  follows 
directly  from  the  definition  given  of  the  MIS  project,  and  is 
related  to  the  point  about  project  orientation.   As  noted,  most 
previous  studies  have  dealt  with  the  total  organization  in 
looking  at  MIS  success,  and  in  doing  so  have  encompassed  an 
indeterminable  amount  of  information  system  development 
activity  that  the  author  would  not  consider  to  be  MIS.   This 
has  had  a  confounding  effect  because  the  requirements  for 
effective  management  of  MIS  activities  appear  to  differ  in 
some  respects  from  the  requirements  for  success  with  non-MIS 
activities.   Specifically,  it  is  the  author's  contention, 
supported  by  the  data  collected  and  analyzed  in  this  thesis, 
that  development  of  information  systems  for  dec is  ion -making  in 
the  various  functional  and  executive  areas  places  a  greater 
premium  on  involvement  and  integration  of  functional  managers 
and  executives  than  does  development  of  non-MIS  computer 
applications,  which  may  have  evolved  fairly  directly  from 
existing  procedures.   This  view  is  supported  by  several  authors 
in  the  field  (Canning  &  Sisson,  1967;  Ditri  &  Wood,  1969; 
Lipperman,  1968;  Reichenbach  &  Tasso,  1968). 

The  last  point  above  has  particular  relevance  when 
viewed  in  the  context  of  the  future  prospects  for  computer  usage 
in  organizations.   Many  organizations  have  about  come  to  the 
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end  of   the  road  on  non-MIS  computer  applications,    and  are, 
therefore,    looking   to   the  opportunities   offered   by   the  more 
sophisticated  and  complex  MIS  applications.      A  report  by 
Scientific  American,    entitled   "The  Computer  Market"    (1968), 
showed    that  only  39. 9%  of   1450  responding  organizations  were 
using  computers    for  MIS   applications   at   the    time  of    the  study, 
compared    to   73%  using  computers    for  accounting  and  payroll 
applications.      However,    the  report  showed   that  907.  of   the 
responding  organizations   planned   to  use  computers    for  MIS 
applications    in   the   future    (p.    17).      Thus,    if   there  are  major 
differences    in   the  organizational   and  procedural   variables 
related   to  success   with  MIS  projects,   as   opposed   to   those 
variables   related   to  success   with  non-MIS  projects,    findings   of 
previous   studies   of   the   total  organization's   MIS   efforts   may 
not  hold  up   in   this  new  MIS   environment.      On   the  other  hand, 
it  may  be    that   the  same  variables  work   in   the  same  directions 
in  both  cases,   or   that   the   findings   of  earlier  studies  may  have 
reflected    the  MIS  part  of   the  application  mix.      Unfortunately, 
the  resolution  of   these  uncertainties    is    impossible  due    to    the 
nature  of    the  data   reported   in  previous   studies. 

Finally,    the  research  reported   in   this    thesis    is   a 
break   from  the    tradition  of   the   intensive  one-case  study  at 
one   extreme,    and    the   large-scale  questionnaire  survey  at   the 
other.      The  approach    in   this   research  has    followed    the   lines 
proposed   by  Mouzelis    (1969),    in   that   a   small   sample  has    been 
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studled  rather  intensively;  not  as  intensively  as  for  a  one- 
case  study,  but  more  intensively  than  is  possible  with  a  wide- 
spread questionnaire  survey.   Further,  the  sample,  although 
small  at  twenty  projects  from  ten  organizations,  represents 
a  diversity  of  organizational  environments,  which  affords  the 
opportunity  for  greater  generalization  than  the  one-case  approach, 

Organization  of  the  Study 
The  author's  prime  objective  in  conducting  the 
research  reported  in  this  thesis  has  been  to  add  to  the  body 
of  knowledge  concerning  the  development  and  implementation  of 
MIS  projects.   Specifically,  an  effort  has  been  made  to 
determine  what  organizational  and  procedural  factors  relate  to 
MIS  project  success.   It  was  recognized  at  the  outset  that  one 
research  effort  could  only  scratch  the  surface  in  a  field  where 
so  little  definitive  research  has  been  done.   However,  it  was 
felt  that  a  well  structured  exploratory  study,  dealing  with 
empirical  data,  could  provide  not  only  answers  to  some  perplex- 
ing questions  in  its  own  right,  but,  more  importantly,  help  to 
define  more  explicitly  those  finer-grain  questions  upon  which 
additional  research  could  be  focused. 

In  pursuing  the  above  objectives,  the  literature  on 
information  systems,  computer  data -processing,  and  project 
management  was  reviewed  to  determine  what  other  authors  have 
had  to  say  that  was  relevant  to  the  subject  at  hand.   The 
results  of  that  review  are  reported  in  Chapter  II. 
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Based  upon  the  literature  survey,  the  author's  own 
experience  in  the  information  systems  field,  and  the  suggestions 
of  faculty  members  and  fellow  graduate  students  at  the  University 
of  Minnesota,  a  research  design  was  gradually  evolved  which  it 
was  believed  would  facilitate  satisfying  the  research  objectives. 
Essentially,  the  approach  involved  defining  a  number  of  hypotheses 
which  could  be  tested  with  empirical  data.   These  hypotheses, 
along  with  the  criteria  of  MIS  project  success,  are  presented 
in  Chapter  III. 

Following  the  enumeration  of  hypotheses  to  be  tested, 
and  the  specification  of  success  criteria,  the  detailed 
methodology  of  the  study  was  worked  out.   The  original  set  of 
hypotheses  was  reduced  to  more  manageable  proportions;  an 
interview  instrument  and  procedures  for  data  collection  were 
developed  and  tested;  the  study  sample  was  selected;  and  the 
method  of  data  analysis  was  decided  upon.   Chapter  IV  deals  with 
these  aspects  of  the  study  methodology. 

The  data  that  were  collected  have  been  presented  in 
two  ways:   Chapter  V  deals  with  the  organization  and  project 
samples  in  descriptive  terms,  which  gives  the  reader  a  "feel" 
for  the  subjects  included  in  the  study;  Chapters  VI  and  VII 
present  the  results  of  statistical  tests  on  the  relationships 
among  the  hypothesis,  criterion,  and  other  variables  for 
which  data  were  collected.   (See  Appendix  E  for  the  list  of 
all  variables  in  the  study.) 
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Chapter  VIII  consists  of  a  summary  of  the  study 
findings,  along  with  some  interpretive  comments  by  the  author. 
Conclusions  drawn  by  the  author,  as  well  as  suggestions  for 
future  research  directions,  are  presented  in  Chapter  EX. 
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CHAPTER  II 
SURVEY  OF  RELEVANT  LITERATURE 

The  survey  of  literature  relevant  to  the  procedural 
and  organizational  variables  relating  to  success  with  MIS 
projects  will  be  developed  along  two  basic  lines:   that 
literature  essentially  prescriptive  in  nature  but  not  founded 
on  empirical  studies,  and  that  literature  reporting  the 
findings  of  empirical  studies. 

Prescriptive  Literature 

The  prescriptive  literature  will  be  considered  in 
two  different  categories.   First  to  be  dealt  with  will  be  the 
literature  pertaining  to  what  can  generally  be  classified  as 
organizational  factors.   This  category  includes  such  subjects 
as  the  location  of  the  MIS  function  within  the  overall  organi- 
zation structure,  the  degree  of  executive  involvement  in  MIS 
development,  the  participation  of  user  departments  in  MIS 
development,  the  selection  and  training  of  MIS  staff,  and  the 
specific  organizational  structure  for  given  projects. 

The  second  category  of  prescriptive  literature  is 
concerned  with  MIS  project  planning  and  control  factors,  and 
is  comprised  of  such  topics  as  definition  of  project  objectives, 
documentation,  allocation  of  manpower  resources  to  projects, 
cost  estimating  of  projects,  the  development  and  application 
of  standards  to  project  personnel,  and  the  use  of  various 
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project  control  techniques  similar  to  CPM  or  PERT. 

Organizational  Factors 

A  topic  that  has  received  a  great  deal  of  attention 
in  the  literature  on  information  systems  is  the  location  in  the 
organization  of  the  information  systems  function.   Diebold  (1964) 
has  contended  that  organizations  have  failed  to  adapt  to  the 
requirements  for  effective  information  systems  by  not  giving 
adequate  attention  to  a  specific  place  in  the  organization 
for  MIS  functions.   Reichenbach  and  Tasso  (1968)  have  pointed 
out  that  the  location  of  the  MIS  function  has  been  determined 
in  many  cases  by  the  nature  of  early  computer  applications, 
not  the  requirements  of  the  current  situation.   Since  early 
use  of  the  computer  was  often  oriented  to  accounting  functions, 
the  computer  activity  was  placed  under  the  controller  and  has 
frequently  been  left  there.   Whitmore  (1966)  has  argued  that 
the  location  of  the  MIS  function  within  the  controller's 
organization  is  a  natural  placement,  in  consonance  with  the 
controller's  overall  responsibilities,  and  should,  therefore, 
remain  there.   Others,  notably  Cole  (1966)  and  Brandon  (1970), 
have  merely  held  that  the  MIS  function  should  be  placed  at  a 
high  level  in  the  organization,  commensurate  with  the  responsi- 
bilities of  the  MIS  manager.   A  counter  point  of  view  to 
Whitmore' s,  expressed  by  Canning  and  Sisson  (1967),  and 
Aukerman  (1966),  is  that  the  MIS  function  should  definitely 
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not  be  under  the  controller  or  a  financial  executive;  rather, 
that  it  should  be  located  directly  under  the  top  executive  as 
an  independent  function.   Finally,  Reif  (1968)  has  reviewed 
the  literature  bearing  on  organizational  placement  of  the  MIS 
function  and  has  provided  an  analysis  of  the  different  points 
of  view.   In  summary,  it  would  appear  that  the  literature  on 
organizational  location  of  the  MIS  function  leans  toward  the 
prescription  of  independent  function  status  at  a  high  organi- 
zational level,  although  this  position  is  not  unanimously  held. 

A  second  organizational  factor  which  has  received 
attention  in  the  literature  is  the  necessity  for  top  management 
and  user  interest  and  participation  in  directing  and  controlling 
the  development  of  MIS  (Aukerman,  1966;  Brandon,  1970;  Canning 
&  Sisson,  1967;  Gallagher,  1961).   The  basic  position  held  in 
common  among  these  writers  is  that  only  through  active  partici- 
pation and  assumption  of  responsibility  for  results  can  execu- 
tives and  managers  expect  to  derive  the  potential  benefits 
from  their  management  information  systems.   In  the  absence  of 
such  positive  direction,  the  MIS  will  end  up  being  what  the 
MIS  staff  provides,  which  may  or  may  not  be  what  is  really 
needed  (probably  the  latter). 

With  respect  to  staffing  the  MIS  function,  Canning 
(1967)  and  Whitmore  (1966),  among  others,  have  prescribed 
means  of  selecting,  developing  and  organizing  the  MIS  staff. 
Campise  (1968)  has  paid  special  attention  to  the  need  for 
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highly  qualified  MIS  project  leaders  who  are  capable  of 
effective  communication  with  user  personnel.   Finally,  Canning 
and  Sisson  (1967),  and  Ditri  and  Wood  (1969)  have  advocated 
the  use  of  inter functional  project  teams  composed  of  MIS  staff 
and  user  personnel  as  a  means  of  achieving  higher  quality  MIS 
products  and  user  organization  acceptance  of  change. 

Project  Planning  and  Control  Factors 

Various  aspects  of  MIS  project  planning  and  control 
have  been  dealt  with  in  the  literature.   Probably  the  best  known 
sources  are  Lecht  (1967)  and  Brandon  (1963;  1968).   Lecht  covers 
virtually  the  entire  range  of  activities  relating  to  project 
planning  and  control,  while  Brandon's  scope  has  been  more 
narrowly  confined  to  the  development  and  application  of  perform- 
ance and  methods  standards.   LaBolle,  et  al  (1965),  working  at 
the  System  Development  Corporation,  have  developed  a  comprehen- 
sive planning  guide  for  computer  project  development  which  also 
deals  with  almost  all  areas  of  project  planning  and  control. 

Schwartz  (1964)  has  stated  that  "Inadequate  planning 
is  at  the  root  of  most  data-processing  failures  and  over-runs 
(p.  14)."   He  has  proposed  using  a  "Work  Breakdown  Structure" 
to  formalize  and  improve  the  planning,  direction,  and  control 
of  projects.   Mensh  (1969)  has  advocated  using  techniques  which 
appear  to  be  quite  close  to  Schwartz's  work  breakdown  structure. 

Blumenthal  (1969)  has  provided  what  is  probably  the 
most  thorough  and  articulate  statement  of  the  way  in  which  an 
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organization  should  approach  its  total  MIS  effort,  as  well  as 
how  specific  projects  should  be  handled.   Blumenthal  advocates 
setting  up  an  analytical  framework  based  on  functional  areas 
of  the  organization.   All  proposals  for  MIS  projects  are  then 
evaluated  within  the  context  of  this  framework.   Blumenthal 
also  provides  a  detailed,  step  by  step  procedure  for  the 
generation,  selection,  development,  implementation,  and  audit 
of  MIS  projects. 

Some  authors  have  recommended  the  use  of  such 
formalized  project  planning  and  control  techniques  as  PERT 
(Canning  &  Sisson,  1967;  Tannert,  1969)  and  PEST  (Program 
Planning  and  Scheduling  Technique,  Anderson,  1966).   The 
difficulty  usually  cited  as  inhibiting  the  utility  of  these 
critical  path  methods  is  the  inability  to  make  accurate 
resource  requirement  and  time  estimates.   Pietrasanta  (1968) 
has  dealt  explicitly  with  this  problem  in  a  discussion  of 
estimates  for  manpower,  machine-time,  money,  and  project  time. 
A  much  more  thorough  treatment  of  this  same  problem,  and  one 
based  on  empirical  data  collected  by  questionnaires  for  169 
computer  programs,  has  been  provided  by  Nelson  (1967). 

Finally,  documentation  methods  and  requirements  for 
MIS  projects  have  been  proposed  by  Snyder  (1965),  Kay  (1969), 
and  Tannert  (1969).   In  fact,  with  regard  to  project  documenta- 
tion, nearly  every  writer  in  the  field  of  MIS  management  has 
lamented  the  pervasive  lack  of  adequate  project  documentation, 
and  exhorted  MIS  management  to  recognize  and  deal  with  this  problem. 
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Empirical  Studies 

As  was  indicated  earlier,  all  of  the  empirical 
research  which  has  been  reported  (or  at  least  found  by  the 
author)  in  the  MIS  area  has  focused  on  the  overall  organization 
rather  than  the  individual  project  (with  the  exception  of  the 
SDC  work  on  programming  costs;  Nelson,  1967).   Most  of  this 
research  has  been  accomplished  either  by  management  consultants 
or  the  American  Management  Association.   The  data  for  these 
various  studies  have  generally  been  collected  by  questionnaire 
or  interview,  in  some  cases  a  combination  of  the  two.   For  ease 
of  presentation,  the  empirical  studies  will  be  discussed  in 
chronological  order. 

Based  on  the  experience  of  124  companies  with  data 
processing  systems,  Baumes  (1961)  drew  several  conclusions  which 
he  put  in  the  form  of  recommendations  for  any  organization 
planning  to  develop  a  computerized  information  system.   The  key 
point  made  by  Baumes  was  the  need  for  detailed  planning  of  the 
tasks  involved  in  information  systems  development.   Such  planning 
should  be  predicated  on  a  clear  statement  of  organizational 
objectives,  and  should  definitely  give  close  attention  to 
behavioral  factors,  as  well  as  technical  factors. 

Thurston  (1962)  has  reported  a  study  of  32  case 
histories  of  organizations'  data  processing  activities.   His 
primary  conclusion  was  that  a  critical  element  in  success  of 
information  systems  was  operating  management's  control  of  systems 
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planning. 

Perhaps  the  best  known  of  the  several  studies  which 
have  been  conducted  by  management  consultants  were  the  two  by 
McKinsey  &  Company  (Garrity,  1963;  Unlocking  the  Computer's 
Profit  Potential,  1968).   Both  of  these  surveys,  based  entirely 
on  interview  data,  were  aimed  at  determining  what  organizational 
and  methodological  factors  had  the  greatest  bearing  on  the 
successful  usage  of  computers  in  the  organizations  studied.   The 
1963  survey  consisted  of  a  sample  of  27  companies  representing 
13  industries.   The  1968  survey  covered  36  companies,  also 
representing  13  industries.   Those  generally  interviewed  were 
the  chief  executive  officer,  the  head  of  data  systems,  and 
other  selected  members  of  management  concerned  with  the  problem 
of  effective  computer  utilization. 

The  conclusions  drawn  from  both  McKinsey  surveys 
were  essentially  the  same.   These  conclusions,  briefly,  were 
the  need  for  top -management  involvement  from  systems  conception 
through  implementation;  the  need  for  expecting  top  performance 
from  the  systems  group  across  a  wide  spectrum  of  applications; 
and  the  need  for  the  participation  of  user  personnel  in  all 
levels  of  systems  design  and  implementation.   Those  companies 
surveyed  which  were  judged  successful  in  their  computer 


This  information  was  provided  by  Mr.  David  B.  Hertz  of 
McKinsey  &  Co.  in  personel  correspondence  with  the  author 
dated  26  November  1968. 
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operations  all  evidenced  these  organizational  and  managerial 
charac  ter  is  tics . 

A  very  general  questionnaire  survey  of  300  business 
firms  using  computers  was  conducted  in  1964  by  Business  Automation 
(Kornblum,  1964).   The  objective  of  this  survey  was  to  find  out 
how  successfully  computer  installations  were  being  used  and 
managed.   It  appeared  that  the  questionnaires  were  completed 
mainly  by  data  processing  personnel  as  the  conclusions  seemed 
to  reflect  the  attitudes  of  the  computer  systems  staff  toward 
others  in  the  organization.   For  the  most  part,  the  opinions 
expressed  indicated  that  those  outside  the  computer  systems 
function  did  not  understand  the  computer  or  how  to  use  it;  they 
did  not  understand  the  unique  problems  and  needs  of  computer 
systems  personnel;  and  they  resented  the  encroachments  of  computer 
systems  in  their  functional  areas. 

The  American  Management  Association  sponsored  a 
survey  in  1965  (Higginson,  1965)  with  the  primary  objective 
of  determining  how  the  companies  studied  were  using  computer 
equipment,  and  the  impact  of  computer  systems  on  their  organiza- 
tions and  operations.   This  survey  was  conducted  both  by  question- 
naire and  interview.   Questionnaires  were  returned   by  288 
companies  of  966  solicited,  and  interviews  were  conducted  in 
21  firms  to  get  more  in-depth  information  as  well  as  to  get  a 
subjective  "feel"  for  the  attitudes  of  management  personnel. 
The  conclusions  were  quite  similar  to  those  already  cited  from 
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earlier  studies:   the  need  for  greater  top -management  direction 
and  control  of  the  computer  systems  function;  the  need  for 
closer  integration  of  staff  specialists  and  operational  manage- 
ment in  the  development  and  evaluation  of  information  systems; 
and  the  need  to  consider  the  computer  systems  function  just  like 
any  other  functional  unit  with  respect  to  organizational 
structure  and  evaluation  of  effectiveness  in  terms  of  corporate 
objectives. 

Two  comprehensive  surveys  have  been  conducted  by 
Booz,  Allen  and  Hamilton,  Inc.,  the  first  in  1966,  and  the  second 
a  year  later  (Computer  Operations  in  Manufacturing  Companies, 
1966;  Computer  Management  in  Manufacturing  Companies,  1967). 
Both  surveys  were  virtually  identical  in  purpose,  varying 
primarily  in  the  sample  sizes  and  means  of  collecting  data. 

In  the  first  Booz,  Allen  &  Hamilton  survey,  the 
stated  purpose  was  "to  determine  what  successful  manufacturing 
companies  have  done,  are  doing,  and  plan  to  accomplish  in 
using  and  managing  the  computer."  The  key  here  was  "successful". 
Only  those  companies  In  the  various  industries  studied  with  the 
fastest  growth  and  greatest  return  on  investment  were  surveyed, 
since  it  was  the  objective  to  determine  how  exceptionally  well 
managed  companies  were  using  the  computer.   The  second  survey, 
conducted  in  1967,  was  different  only  to  the  extent  that  a  wider 
range  of  data  covering  an  expanded  sample  was  sought. 
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The  1966  survey  was  conducted  entirely  by  Interview, 
using  no  questionnaire  as  far  as  can  be  ascertained.   Thirty- 
three  companies  were  included  in  the  survey,  and  189  interviews 
were  conducted  within  them.   Over  half  of  the  in-depth  inter- 
views were  conducted  with  top  and  operating  management.   The 
findings  were  then  evaluated  in  an  attempt  to  correlate  computer 
usage  and  management  with  success  of  the  company  in  question. 
The  1967  survey  used  a  sample  of  108  manufacturing  companies, 
and  questionnaires  were  employed  to  collect  the  data.   The 
questionnaires  covered  organizational  structure  and  practices 
In  each  company,  as  well  as  specific  information  on  computer 
costs,  usage  and  plans. 

The  conclusions  drawn  from  both  surveys  were  mainly 
enumerations  of  the  various  composite  computer  management 
characteristics  for  successful  companies.   The  reports  did 
indicate  that  a  trend  was  developing  toward  more  top  management 
and  operational  management  involvement  in  computer  systems  plans 
and  audits.   Further,  it  appeared  that  the  trend  was  to  move 
the  overall  responsibility  for  computer  activities  away  from 
financial  executives  to  a  computer  executive  at  top  corporate 
level. 

A  questionnaire  survey,  conducted  in  1967  as  part  of 
the  Diebold  Research  Program  (Summary  Report  of  a  Survey  on 
the  Cost  Effectiveness  of  Software  &  Hardware,  1967),  was 
designed  to  explore  the  future  management  implications  of 
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computer  information  systems,  and  to  develop  some  standard 
measurements  of  the  cost  effectiveness  of  hardware  and  software. 
The  respondents  to  the  questionnaire  represented  177.  of  those 
solicited,  and  consisted  of  senior  information  systems  manage- 
ment, technical  staff  reporting  to  information  systems  manage- 
ment, and  top  corporate  management.   The  findings  of  relevance 
here  were  that  in  the  majority  of  cases  top  management  was  not 
active  in  guiding  the  growth  of  information  systems  activities, 
and  that  new  computer  applications  were  often  proposed  by 
information  systems  management  rather  than  those  closer  to  the 
application  area,  namely,  user  management.   The  major  problem 
with  proper  utilization  of  computer  information  systems  was 
found  to  be  the  lack  of  concern  on  the  part  of  management  for 
maximum  utilization.   It  was  also  concluded  that  larger  computer 
users  were  more  likely  to  have  budget  and  control  procedures 
for  information  systems  activities,  and  were  generally  getting 
greater  return  on  their  information  systems  investments  than 
smaller  users. 

Lipperman  (1968)  has  reported  the  results  of  investi- 
gations in  twelve  organizations  using  computer  information 
systems.   His  data,  collected  by  interview,  were  organized  on 
a  case  basis  to  portray  what  various  types  of  organizations 
have  done  to  successfully  Implement  computer  systems.   Lipperman 
dealt  with  both  "operative"  systems  and  planning  systems,  and 
pointed  to  the  latter  as  the  direction  in  which  information 
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sys terns  should  develop  as  operative  subsystems  are  integrated 
at  higher  levels.  This  particular  study  would  seem  to  be  the 
first  dealing  specifically  with  MIS  as  defined  in  this  thesis. 

Several  points  were  made  by  Lipperman  which  are  of 
special  interest  here.   As  organizations  moved  toward  higher 
level  systems  that  integrated  information  from  various  subsystems, 
more  user  management  responsibility  for  defining  system  require- 
ments was  essential.   Following  from  this  was  the  need  to 
locate  the  information  systems  function  high  in  the  management 
hierarchy,  independent  of  such  other  functional  executives  as 
the  controller  or  financial  vice-president. 

A  study  undertaken  for  the  American  Management  Associa- 
tion, under  the  direction  of  Reichenbach  and  Tasso  (1968), 
was  based  on  the  hypothesis  that  "the  location  of  the  computer 
complex  within  the  organization  will  affect  the  complex's  opera- 
tions (p.  16)."  This  study  consisted  of  in-depth  interviews 
with  three  levels  of  management  in  sixteen  companies  representing 
nine  industries.   The  organizational  members  interviewed  were 
top  executives,  information  systems  personnel,  and  user  organi- 
zation managers.   Although  numerous  conclusions  were  drawn 
from  the  sixteen  case  studies,  the  gist  of  the  findings  can  be 
summarized  briefly  as  follows: 

The  location  of  the  computer  complex  does  have  a 
very  real  relationship  with  its  effectiveness  In 
meeting  organization  needs.   (No  evidence  was 
provided  on  the  measure  or  meaning  of  effectiveness.) 
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Th  e  computer  information  systems  function 
should  report  directly  to  top  management, 
and  should  not  be  a  part  of  any  existing 
traditional  function. 

The  centralization  of  the  computer  activi- 
ties need  not  have  any  impact  on  the 
traditional  form  of  the  corporate  structure. 
The  latter  is  a  function  of  management's 
philosophy  and  objectives. 

The  nature  of  an  organization's  basic 
activities  and  processes  bears  on  the 
ease  of  its  development  of  computer  infor- 
mation systems. 

Participation  by  top  level  and  user  manage- 
ment in  the  development  of  information 
systems  will  enhance  acceptance  of  those 
systems,  improve  utilization  of  the  compu- 
ter, and  provide  results  more  responsive 
to  management's  needs. 

A  monograph  by  Reif  (1968)  reported  the  findings  of 
research  he  conducted  for  his  doctoral  dissertation  in  1966. 
The  literature  survey  comprising  Chapter  2  of  this  monograph 
has  been  cited  previously.   The  focus  here  will  be  on  Reif's 
research  methodology  and  findings.   The  impetus  behind  Reif's 
research  was  his  belief  that  a  major  obstacle  facing  organiza- 
tions attempting  to  achieve  maximum  benefits  from  computer 
systems  was  the  conflict^  between  the  imperatives  of  computer 
technology  and  the  traditional  structure  of  business  organizations 

"There  is  a  growing  number  of  we 11 -documented 
studies  which  show  that  the  two  are  not  com- 
patible; yet  most  firms  today  are  content  to 
achieve  limited  results  by  super -imposing 
advanced  information  systems  on  organizational 
structures  which  are  unable  to  adjust  to  the 
new  requirements  and  interpersonal  relation- 
ships dictated  by  the  systems  design  (from 
the  Preface,  p.  iv)." 
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The  specific  purpose     of  Reif's   study  was   "to  examine  what 
structural   changes   occur   following   the    implementation  of 
computer  systems    in  business   organizations    (p.    41)."     This 
purpose  was   enumerated    into   several   explicit   factors   3uch  as 
the   locus   of  decision  making  within   the  management  hierarchy, 
formal   and   informal  communication  channels,   and    the   functional 
integration  of  organizational  activities.      In  addition,   Reif 
studied   the  role  of   the  computer  systems   staff,    its    location 
in   the  organization,   and   its   relationships  with  user  departments 

Reif  used   the  case  method   for  studying   three   firms: 
a  utility  company,   a  bank,   and  a  manufacturing   firm.      Most  of 
the  data  were  collected  by   interviews,   both  structured  and 
unstructured.      The  remainder  of   the  data  came   from  company 
records.      Before  considering  Reif's    findings,    it   is    important 
to  note   that  his   data  reflect  a  considerable  number  of  non-MIS 
computer  applications,   what  might  be   termed   "production"   jobs, 
as   opposed   to    the  kind  of  decision  oriented  management   informa- 
tion which    is    the   focus  of   the  current  research. 

Based  on  his   research,    some  of  Reif's   conclusions 


were: 


The  presence  of  the  computer  resulted  in 
centralizing  dec  is ion -making  within  the 
management  hierarchy. 

Line-staff  relationships  were  changed. 
The  computer  staff  group  exerted  great 
influence  and  made  decisions  affecting 
the  internal  activities  of  operating 
departments.  This  staff  influence  was 
de  facto,  not  usually  legitimized  by 
formal  organization  structure. 
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Lateral  flows  of  communication  throughout 
the  management  hierarchy  were  formed  and 
expanded.   However,  formal  channels  of 
communication  were  not  generally  revised. 

The  impact  of  computer  systems  will  be 
greatest  on  middle  management  as  routine 
administration  and  control  decisions 
become  programmed;  these  positions  will 
generally  decrease  in  number.   However, 
the  impact  on  middle  managers  will  vary, 
depending  on  the  organization,  top 
management's  objectives,  and  the 
functional  area  concerned. 

The  hierarchical  structure  will  be 
retained  in  organizations  but  "certain 
changes  in  the  management  framework 
appear  to  be  imperative  if  the  firm 
is  to  maintain  harmony  between  infor- 
mation technology  and  the  structure 
of  organization  (p.  114)." 

In  an  attempt  to  measure  the  effectiveness  of  manage- 
ment information  systems  in  terms  of  profitability,  a  questionnaire 
survey  was  initiated  by  S.  D.  Leidesdorf  &  Co.  in  1967  (The 
Profitability  of  Management  Information  Systems,  1969).   Of  142 
companies  requested  to  participate  in  this  survey,  130  did  so. 
These  130  companies,  representing  eleven  industry  groupings, 
were  known  to  be  using  computers,  "presumably  effectively." 

Although  the  objective  of  the  Leidesdorf  study  dealt 
specifically  with  the  terms  MIS  and  profitability,  the  use  of 
these  terms  may  be  somewhat  misleading.   The  profitability 
measure  was  derived  by  asking  the  participating  companies  "to 
express  as  a  percentage  the  influence  of  their  computer  operations 
and  associated  expenses  on  profitability  during  their  first  five 
years  (or  less,  where  that  was  appropriate)  of  computer-based 
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data  processing  (p.  18)."   This  approach  raises  two  issues: 
first,  from  information  contained  in  the  study  report,  it  is 
impossible  to  determine  what  data  went  into  each  company's 
response  on  profitability,  or  how  that  data  was  aggregated 
to  make  the  requested  computation;  and  second,  since  the 
requested  computation  dealt  with  the  early  period  of  each 
company's  computer  use,  it  is  highly  probable  that  return  from 
MIS  applications  was  greatly  confounded,  if  not  overshadowed, 
by  return  from  non-MIS  applications.   This  latter  observation 
is  made  in  view  of  the  tendency  for  organizations  installing 
computers  to  implement  cost-displacement  applications  first, 
with  MIS  applications  following  as  more  experience  and  sophis- 
tication are  acquired. 

With  respect  to  the  results  of  the  Leidesdorf  study, 
32  of  the  130  companies  were  viewed  as  being  successful  with 
their  MIS  in  terms  of  the  reported  percentage  profitability 
contribution  of  MIS.   The  study  concluded  that  these  32 
companies  achieved  the  high  MIS  contribution  to  profitability 
by  "intelligent  planning  and  control".   For  the  98  companies 
viewed  as  not  successful  in  terms  of  MIS  contribution  to 
profitability,  the  report  stated  that: 

"The  common  denominator  seems  to  be  defi- 
ciencies in  planning  associated  with  the 
operation.   To  a  lesser  extent,  the  absence 
or  incompetence  of  functional  and/or  execu- 
tive participation  in  the  planning  and 
operation  of  the  project  appears  to  be  a 
contributing  factor  (p.  21)." 
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To  summarize  the  Leidesdorf  study,  it  was  concluded 


that: 


"Management  information  systems  represent 
a  favorable  field  for  investment  and  can 
make  a  real  contribution  to  prof itability-- 
especially  through  improved  control  of  the 
business;  but  positive  results  are  not 
automatic  and  guaranteed.   Serious  problems 
can  and  do  arise  and  are  usually  attribut- 
able to  deficiencies  in  management  involve- 
ment, planning,  and  control  (p.  24)." 

Before  concluding  this  section  on  the  survey  of  the 
literature,  two  doctoral  dissertations  warrant  mention.   Holsinger 
(1970)  studied  the  constraints  to  implementation  of  data  process- 
ing systems,  and  examined  the  roles  of  key  decision  makers 
involved  in  implementing  such  systems.   Holsinger  obtained  his 
data  through  interviews  with  members  of  top  management,  key 
technical  personnel,  and  "third  party  representatives."  The 
constraining  variables  were  categorized  as  technological,  organi- 
zational, and  individual.   Holsinger  concluded  that  the  relative 
importance  of  these  three  categories  has  shifted  over  time; 
that  as  size  increases,  the  importance  of  technology  and  organi- 
zation increases  at  the  same  time  that  the  importance  of  the 
individual  decreases;  that  size  influences  the  organization's 
ability  to  utilize  a  proper  division  of  labor;  that  control 
problems  were  created  through  the  determination  by  technologists 
of  critical  aspects  of  systems  design;  and  that  structural 
problems  of  integrating  systems  functions  with  various  types  of 
information  systems  appeared. 
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The  second  dissertation,  by  Hcdgetts  (1968),  is  of 
interest  since  it  deals  with  project  management.   Based  on  the 
research  he  conducted,  Hodgetts  advocates  the  project  approach 
as  opposed  to  the  functional  approach  where  an  organization  is 
confronted  with  managing  a  specific,  complex  task  with  stringent 
quality  parameters.   He  enumerates  the  pros  and  cons  of  the 
project  management  approach  and  points  out  that  it  is  a  manage- 
ment tool  not  an  organizational  panacea.   Of  particular  interest 
are  his  observations  concerning  the  interface  problems  between 
the  project  organization  and  the  permanent  organization,  and 
the  attributes  of  effective  project  managers.   Although  Hodgetts 
was  concerned  primarily  with  project  management  in  the  aero- 
space, construction,  chemical,  and  consumer  products  industries, 
and  not  with  MIS  projects,  his  inter-industry  approach,  the 
focus  on  individual  projects  and  the  evaluation  of  project  tech- 
niques and  relationships  are  relevant  to  the  present  research. 

Summary 
Although  the  literature  relevant  to  management  infor- 
mation systems  is  rather  extensive,  there  are  few  reports  of 
well  conceived  and  structured  research  in  the  field.   While 
prescriptions  and  definitions  have  been  plentiful,  they  have 
seldom  been  supported  by  data.   Numerous  authors  have  presented 
their  views  on  virtually  all  aspects  of  information  systems, 
but  these  views  have  generally  stemmed  from  their  authors' 
personal  experiences  rather  than  structured  research.   Those 
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research  studies  which  have  been  reported  have  tended  to  be 
broad-brush.   They  have  generally  represented  wide-scale 
questionnaire  and  interview  surveys  which  were  aimed  at  overall 
organizational  use  of  computers,  or  case  studies  of  the  impact 
of  computer  usage  on  organization  structures  and  relationships. 

The  above  comments  concerning  the  literature  relevant 
to  MIS  are  not  intended  as  criticism  of  what  others  have  done. 
Rather,  the  point  to  be  made  is  that  in  a  field  growing  so 
rapidly,  with  so  few  definitive  guidelines,  a  great  deal  more 
needs  to  be  done.   Researchers  and  practitioners  in  the  MIS 
field  must  define  areas  to  be  researched,  then  develop  well- 
conceived  approaches  to  conducting  such  research.   What  has 
already  been  accomplished  will  be  helpful,  as,  indeed,  it  has 
been  helpful  to  the  author  in  conceiving  and  designing  the 
research  reported  in  this  thesis.   But  the  overwhelming  impression 
one  is  left  with  after  reviewing  the  information  systems  litera- 
ture is  the  great  gap  between  what  is  needed  and  what  has  been 
accomplished. 

Building  on  what  others  have  said  and  done,  the 
author  has  made  an  attempt  to  start  closing  that  gap.   The 
prescriptions  and  empirical  findings  of  others  were  major 
contributions  to  the  definition  of  the  hypotheses  posited  and 
tested  in  the  present  research.   The  techniques  others  have 
employed  have  been  invaluable  guides  to  the  author  in  designing 
the  data  collection  methodology  used  in  this  research. 
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Nonetheless,  the  research  conducted  by  the  author,  and  reported 
in  this  thesis,  represents  a  departure  from  previous  research 
in  both  approach  and  substance.   Rather  than  collecting  a  mass 
of  data,  then  setting  up  hypotheses  to  be  tested,  the  author 
of  the  present  study  set  up  the  hypotheses,  then  developed  data 
collection  and  analysis  techniques  to  test  them.   As  opposed 
to  looking  at  overall  computer  usage  in  many  organizations,  or 
the  intensive  study  of  one  or  two  organizations  to  determine 
what  impact  computer  usage  has  had  on  them,  the  present  research 
has  focused  on  the  MIS  project.   Specific,  measurable  criteria 
of  project  success  have  been  defined  in  order  that  the  hypotheses 
posited  could  be  tested. 
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PART  II 

HYPOTHESES,  CRITERIA  OF  SUCCESS,  AND 
METHODOLOGY  OF  THE  STUDY 

This  part  of  the  thesis  deals  with  the  means  by  which  it  was 

attempted  to  formulate  answers  to  the  question  posed  earlier: 

"What  organizational  and  procedural  factors 
are  correlates  of  success  with  MIS  projects?" 
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CHAPTER  III 
HYPOTHESES  AND  CRITERIA  OF  SUCCESS 

Introduction 

To  assist  in  identifying  the  organizational  and 
procedural  factors  related  to  success  with  MIS  projects,  a 
list  of  thirty-five  hypotheses  was  developed.   These  hypotheses 
were  constructed  from  the  literature  previously  reviewed,  from 
the  author's  personal  experience,  and  from  discussions  with 
various  professionals  in  the  MIS  field.   The  hypotheses  provided 
a  framework  for  the  collection  and  analysis  of  data  relevant 
to  the  purpose  stated  above. 

In  addition  to  the  list  of  hypotheses,  a  set  of 
criteria  was  developed  to  evaluate  relative  success  of  given 
MIS  projects.   The  criteria  will  be  taken  up  first,  followed 
by  a  discussion  of  the  hypotheses. 

Criteria  of  Success 
A  given  MIS  project  can  be  viewed  from  several 
perspectives  in  attempting  to  judge  whether  or  not  it  was 
successful.   It  is  necessary  that  the  criteria  selected  be 
relevant,  be  as  objective  as  possible,  and  at  the  same  time  be 
realistic  in  terms  of  ease  and  consistency  of  measurement.   Four 
separate  criteria  which  appear   to  meet  these  requirements  are: 
1.   Success  in  terms  of  time.   How  closely 
did  actual  time  spent  on  the  project 
conform  to  the  time  allocated  for  it? 
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2.  Success  in  terms  of  development  cost. 
Was  the  project  completed  within  the 
budgeted  cost?   If  over-runs  were 
experienced,  what  was  the  total  cost 
as  a  percentage  of  budgeted  cost? 

3.  Success  in  terms  of  users'  evaluations. 
Did  the  completed  project  satisfy  user 
requirements  and  expectations?  Were 
there  unanticipated  problems  or  benefits 
resulting  from  implementing  the  completed 
project? 

4.  Success  in  terms  of  computer  operations. 
Has  the  completed  project  created  problems 
for  computer  operations  in  scheduling,  set 
up,  run  time,  or  control? 

It  is  believed  that  the  above  criteria  are  relevant 
operational  measures  of  project  success,  and,  as  such,  provided 
a  suitable  means  of  testing  the  hypotheses  selected  for  evaluation, 

Hypotheses 
The  thirty-five  factors  which  were  hypothesized  to  be 
related  to  MIS  project  success  fall  into  the  following  broad 
categories : 

1.  Characteristics  and  competence  of  project 
personnel. 

2.  Project  management  and  control  techniques. 
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3.  Organizational  Interaction  factors. 

4.  Project  specific  factors. 

5.  Global  factors  comprising  the  project 
environment. 

The  hypotheses  were  not  viewed  as  exhaustive  nor  were  their 

assignments  to  categories  clear  cut  in  all  cases.   However,  it 

was  felt  that  both  the  hypotheses,  and  the  categories  into  which 

they  were  placed,  provided  a  suitable  framework  for  conducting 

research. 

The  hypotheses  are  presented  below  by  categories, 

with  a  brief  description  of  what  each  category  represents.   The 

individual  factors  or  hypotheses  should  be  self-explanatory. 

In  order  to  avoid  redundancy  in  the  list  of  hypotheses,  an 

example  of  the  form  for  each  full  hypothesis  will  be  given 

here.   The  list  itself  will  merely  consist  of  the  body  of  the 

given  hypothesis  in  the  form  of  a  factor  ostensibly  related  to 

MIS  project  success. 

Example:   It  is  hypothesized  that  the  higher 
the  level  of  formal  education  of 
project  personnel,  the  greater  the 
likelihood  of  MIS  project  success. 

Characteristics  and  Competence  of  Project  Personnel  (Category  I) 

This  category  should  require  little  explanation.   It 
is  merely  a  grouping  of  those  hypotheses  about  the  competence 
of  people  who  worked  directly  on  the  project.   While  the  general 
statement  "the  more  competent  the  personnel  working  on  the  project, 
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the  more  likely  success"  is  something  of  a  truism,  the  factors 
falling  under  competence  may  be  debatable  in  certain  cases. 
Such  factors  are,  then,  testable. 

1.  Coordinating  ability  of  project  leader. 

2.  Systems  experience  of  project  personnel. 

3.  Persuasiveness  of  project  leader  (as 
evaluated  by  superiors). 

4.  Proficiency  of  project  personnel  (as 
judged  by  superiors). 

5.  Low  turnover  of  project  personnel. 

6.  Length  of  experience  in  the  organiza- 
tion of  project  personnel. 

7.  High  formal  educational  level  of 
project  personnel. 

Organizational  Interaction  Factors  (Category  II) 

This  category  includes  those  hypotheses  dealing  with 
how  the  user  organization  interfaces  with,  supports,  and  is 
supported  by  the  MIS  department.   It  is,  in  short,  a  group  of 
hypotheses  focused  on  the  integration  of  the  overall  organiza- 
tion with  respect  to  development  and  utilization  of  management 
information  systems. 

8.  Participation  by  operating  management  in 
design,  formal  approval  of  specifications, 
and  continual  review  of  project. 
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9.   Utilization  of  a  project  team  composed 
of  MIS  staff  and  user  personnel. 

10.  Operating  management  conducts  periodic 
management  audit  of  MIS  function. 
(Evaluation  of  effectiveness  for  users). 

11.  Organizational  level  of  top  computer 
executive. 

12.  Formal  training  program  set  up  for 
user  organization. 

13.  Project  originated  by  user  organization. 

Project  Management  and  Control  Techniques  (Category  III) 

Subsumed  within  this  category  are  those  hypotheses 
pertaining  to  specific  methods  used  to  plan,  direct,  monitor, 
and  control  a  given  project.   This  group  represents  many  of 
what  are  frequently  prescribed  as  "good  practices"  in  the 
literature  on  management  of  the  data  processing  function. 

14.  Measurable  project  objectives  from 
conception  of  the  project. 

15.  Formal  project  selection  process  used 
to  determine  which  projects  to  develop. 

16.  Documentation  standards  used  and  enforced. 

17.  Use  of  a  formalized  and  regular  reporting 
structure  on  project  progress. 

18.  Planning  and  accounting  for  all  resources 
throughout  project  development. 
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19.  Program  maintenance  and  review  responsibility 
specified  for  definite  period  after  implementation. 

20.  Utilization  of  a  formal  time-scheduling 
technique  such  as  PERT  for  project  development. 

21.  Performance  standards  employed  for  analysts 
and  programmers. 

Project  Specific  Factors  (Category  IV) 

This  category  encompasses  the  various  factors  that 
are  unique  to  a  given  MIS  project,  such  as  the  programming 
language  used. 

22.  High  availability  of  computer  time  for 
program  testing. 

23.  High  level  programming  language  used 
for  project. 

24.  Utilize  existing  data  base  versus 
constructing  or  greatly  modifying  one. 

25.  Short-term,  minor  project  versus  large, 
complex  project. 

Global  Factors  Making  up  the  Project  Environment  (Category  V) 

The  global  category  covers  a  rather  wide  range  of 
factors  as  the  name  would  imply.   These  are  environmental 
variables  which  might  be  viewed  as  more  indirect  in  effect  than 
the  factors  under  other  classifications.   In  a  sense  this  is  a 
sort  of  omnibus  category  that  includes  the  hypotheses  that  were 


-41- 

considered  relevant  but  did  not  easily  fit  into  a  more  specific 
grouping. 

26.  High  centralization  of  organizational 
MIS  activities. 

27.  Number  of  years  experience  for  organiza- 
tion with  computerized  information  systems. 

28.  Low  turnover  rate  of  MIS  staff. 

29.  Combination  analyst/programmers  for 
small  projects. 

30.  High  rates  of  MIS  staff  drawn  from 
within  the  organization. 

31.  High  average  income  level  of  MIS  staff. 

32.  Low  degree  of  overall  organizational 
change. 

33.  Separation  of  analysts  and  programmers 
for  large  projects. 

34.  Overall  size  of  organization  systems  staff. 

35.  Ratio  of  computer  hardware  investment  to 
total  sales  or  operating  budget. 

Summary 
This  chapter  has  been  devoted  to  presenting  the  factors 
which  were  hypothesized  to  be  related  to  MIS  project  success  and 
to  defining  specific  criteria  of  success  suitable  for  testing 
those  hypotheses.   Chapter  IV  details  the  methods  which  were  used 
to  test  the  hypotheses. 
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CHAPTER  IV 
METHODOLOGY  OF  THE  STUDY 

Initial  Evaluation  of  the  Hypotheses 
The  above  list  of  thirty-five  hypotheses  would  impose 
a  considerable  burden  on  anyone  desiring  to  test  them  in  field 
research.   For  this  reason,  it  was  felt  advisable  to  try  to 
sort  out  the  various  hypotheses  into  some  kind  of  priority 
hierarchy.   The  result  of  such  a  sorting  process  would  be  a 
ranking  of  all  the  factors,  from  those  viewed  as  most  crucial 
to  project  success  to  those  viewed  as  least  significant.   It 
was  decided  that  the  best  means  of  coming  up  with  such  a 
priority  ranking  of  the  hypotheses  was  to  get  the  opinions  of 
professionals  in  the  MIS  field. 

The  opportunity  for  getting  the  opinions  of  MIS 
professionals  on  the  importance  of  the  various  hypotheses  to 
project  success  was  afforded  by  the  Founding  Conference  of  the 
Society  for  Management  Information  Systems  (SMIS),  held  at  the 
University  of  Minnesota  in  late  1969.   The  decision  was  made  to 
submit  the  hypotheses  in  questionnaire  form  to  the  attendees  of 
this  conference.   The  expected  product  of  the  completed  question" 
naires  was  to  be  the  desired  priority  ranking. 

Pre-questionnaire 
To  facilitate  developing  a  questionnaire  which,  when 
completed,  could  be  evaluated  in  such  a  way  as  to  provide  the 
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ranklng  by   Importance  of   the  hypotheses,    it  was   decided    that  a 
pre-questionnaire  should  be  prepared  and  submitted   to  a  group 
of  MIS  professionals    in   the  Twin  Cities   area.      The  respondents 
(N=43)    for   the  pre-questionnaire  were  all   employed  by   the   firms 
associated  with   the  Management   Information  Systems  Research 
Center  of   the  University  of  Minnesota. 

The  objective   in  preparing  and  evaluating   the  pre- 
questionnaire  was    to   come  up  with  a   "neutral"  hypothesis  which 
could   be  used  as   a  reference  or  anchor  point   in  constructing   the 
final   questionnaire   to  be  submitted   to    the  SMIS  conference 
participants.      This   neutral   hypothesis  was    to  be   the  basis   of 
comparison   for  each  of   the  remaining   thirty-four  hypotheses. 
By  choosing  a   "neutral"  hypothesis   as    the  standard   for   comparison, 
one  seldom  viewed  as   either  most   Important  or   least   important 
to  MIS  project  success   by   the  pre-questionnaire  respondents, 
it  was   expected    that   the  desired  spread    in  priority  rankings 
would  be  derived   from   the   final   questionnaire. 

The  procedure  employed   for   the  pre-questionnaire  was 
to   list   the   thirty-five  hypotheses,   or   factors,   disregarding 
any  classification  by  categories.      Opposite  each  hypothesis 
were    two   blanks,   one  under  a  column  headed   "Most   Important" 
and    the  other   under  a   column  headed   "Least    Important".      The 
respondent  was    then  requested   to   evaluate   each  of   the   thirty- 
five   factors   as   to   its   criticality   for  MIS  project  success, 
and   to   indicate  by   the  numbers    1    to   7    those  seven   factors 
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considered   to  be  most   important,   and   those  seven   factors 
considered   least   important,    in  contributing   to  project  success. 
(See  Figure  1). 

FIGURE   1 
Pre-ques  tionnaire 

Most  Least 

Factor  Important  Important 

1)  High   formal   educational    level 

of  project  personnel  

2)  Proficiency  of  project  personnel 

as  judged  by  superiors  


35)      Etc.  Etc.         Etc. 

The  returned  pre-questionnaires  were  analyzed    to 
determine  which   factor  was   "most  neutral".      That   is,   which 
factor  was   regarded    to  be  of  middle   importance   in   the  range  of 
thirty-five.      This    factor  was,   of  course,   one    that  was   seldom 
ranked  as   one  of   the  seven  most  or  seven   least   important   in 
contributing    to  MIS   project  success. 

SMIS  Questionnaire 
Analysis   of    the  responses   showed    the   "most  neutral" 
factor    to   be:      "Performance   standards   employed    for   analysts 
and   programmers".      This    factor  was    then  used    to  construct    the 
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final  questionnaire    in    the   following  manner.      The  remaining 

thirty-four   factors  were   listed  as   before  with  no   categorization. 

Opposite  each   factor  was   a  Likert-type  scale  allowing  for  one 

of   five  responses    to  each   item.      At   the   top  of  each  page  of   the 

questionnaire    the  neutral    (standard)    factor  was   printed.      The 

respondent  was    then  asked   to  rate  each  of   the   thirty-four 

remaining   factors    for    importance   in  contributing   to  MIS  project 

success   by  comparing   it   to   the   standard   factor.      The   form  of 

the  response  was   merely  a  check   in   the  chosen  one  of    the   five 

blanks.      In  addition,    each  respondent  was   provided  space  at   the 

end  of   the  questionnaire   to  write   in  any  other   factors  which 

he  believed   important   to  MIS  project  success,   but  which  were 

not   included    in   the   list  of   thirty-four   factors    to  be  rated 

(see  Appendix  A). 

Of  the  roughly  250  participants  in  the  SMIS  conference, 

142  returned  the  questionnaire.   The  returned  questionnaires 

were  tallied  by  determining  the  number  of  responses  in  each  of 

the  five  levels  for  each  factor.   Each  level  was  then  given  a 

numerical  value  as  follows: 

Much  less  1 

Somewhat  less  2 

About  same  3 

Somewhat  more  4 

Much  more  5 

Multiplying  the  number  of  responses  per  level  by  the 

level  value,  summing  across  all  levels  for  a  factor,  then 

dividing  by  the  number  of  responses  for  that  factor,  yielded  a 
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numerical  score  for  each  factor.   The  factors  were  then  ordered 
on  the  basis  of  these  scores  to  give  a  ranking  from  most 
important  to  least  important. 

Results  from  the  SMIS  Questionnaire 
Evaluation  of  the  returned  questionnaires  resulted  in 
the  factor  ranking  shown  in  Figure  2.   The  scores  for  the 
individual  factors  ranged  from  4.38  for  the  factor  judged  most 
important  for  MIS  project  success  to  2.30  for  the  least  important 
factor.   In  addition,  as  will  be  noted  in  Figure  2,  the  variance 
was  calculated  for  each  factor.   These  variances,  ranging  from 
.602  for  the  factor  ranked  second  in  importance  to  1.519  for 
the  factor  ranked  seventeenth,  were  computed  to  determine  the 
relative  agreement  among  respondents  on  the  factors,  and  were 
not  used  for  any  purpose  beyond  that. 

Analysis  of  SMIS  Questionnaire  Data 

As  was  pointed  out  earlier,  the  questionnaire  was 
constructed  and  administered  to  facilitate  field  study  of  the 
hypotheses.   The  ranking  provided  a  means  of  focusing  on  what 
factors  those  active  in  the  MIS  field  believed  to  be  the  most, 
the  least,  and  of  intermediate  importance  to  project  success. 
The  next  steps  were  to  select  the  hypotheses  to  be  further 
evaluated,  and  to  design  a  means  of  collecting  data  in  organi- 
zations on  both  the  hypotheses  and  the  criteria. 
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FIGURE  2 
Ranking  of  Hypotheses 


Computed 
Rank  Score    Variance 

*  1  Participation  by  operating  management  (\  w  x  /N) 

in  design,  formal  approval  of  spec  if i-  ^ — ■ 

cations  and  continual  review  of 

project.  4.38       .676 

*  2  Measurable  project  objectives  from 

conception  of  the  project.  4.29       .602 

*  3  Utilization  of  a  project  team  composed 

of  MIS  staff  and  user  personnel.         4.25       .643 

4  Coordinating  ability  of  project  leader.    4.21       .620 

5  Operating  management  conducts  periodic 
management  audit  of  MIS  function. 

(Evaluation  of  effectiveness  for  users).   4.00       .970 

6  Formal  training  program  set  up  for 

user  organization.  3.95       .872 

*  7  Organizational  level  of  top  computer 

executive.  3.94      1.004 

*  8  Systems  experience  of  project  personnel.   3.90       .642 

9  Formal  project  selection  process  used 

to  determine  which  projects  to  develop.    3.88      1.058 

10  Persuasiveness  of  project  leader 

(superior's  evaluation).  3.84       .856 

11  Proficiency  of  project  personnel 

(as  judged  by  superiors).  3.79       .780 

*12  Documentation  standards  used  and 

enforced.  3.74       .942 

*13   Use  of  a  formalized  and  regular  report- 
ing structure  on  project  progress.        3.68       .686 

*14  Low  turnover  of  project  personnel.        3.56       .862 

*  Hypotheses  selected  for  further  study. 


-48- 


Computed 
Score 

Variance 

3.50 

.968 

3.48 

1.237 

3.46 

1.519 

FIGURE  2  (Cont.) 


Rank 

15  Planning  and  accounting  for  all 

resources  throughout  project  develop- 
ment. 

*16  Source  of  origination  of  project  (MIS 
staff  or  user). 

*17  High  centralization  or  organizational 
MIS  activities. 

18  Program  maintenance  and  review 
responsibility  specified  for  definite 

period  after  implementation.  3.45     1.073 

19  Number  of  years  experience  for 
organization  with  computerized 

information  systems.  3.43      .934 

*20  Length  of  experience  in  the  organiza- 
tion of  project  personnel.  3.32     1.051 

21  Utilization  of  a  formal  time -scheduling 
technique  such  as  PERT  for  project 
development. 

22  High  availability  of  computer  time  for 
program  testing. 

*23  High  level  programming  language  used 
for  project. 

24  Utilize  existing  data  base  versus 
constructing  or  greatly  modifying  one. 

25  Low  turnover  rate  of  MIS  staff. 

26  Short-term,  minor  project  versus 
large,  complex  project. 

27  Combination  analyst/programmer  for 
small  projects. 


*  Hypotheses  selected  for  further  study. 


3.25 

1.056 

3.04 

1.272 

3.00 

1.059 

2.99 

1.511 

2.91 

1.029 

2.89 

1.437 

2.86 

1.191 

-49- 


FIGURE  2  (Cont.) 


Computed 
Rank  Score   Variance 

28  High  rates  of  MIS  staff  drawn  from 

within  the  organization.  2.81     1.104 

29  High  average  income  level  of  MIS  staff.    2.77      .750 

30  Low  degree  of  overall  organizational 

change.  2.72     1.263 

*31  High  formal  educational  level  of 

project  personnel.  2.71     1.109 

*32  Separation  of  analysts  and  programmers 

for  large  projects.  2.51     1.274 

*33  Overall  size  of  organization  systems 

staff.  2.50     1.207 

*34     Ratio  of  computer  hardware   investment 

to   total  sales  or  operating  budget.  2.30  1.188 


*   Hypotheses   selected   for   further  study. 
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Two  approaches  were  used  in  selecting  the  hypotheses 
for  further  evaluation.   First,  it  was  desired  that  about  three 
factors  be  evaluated  from  the  top,  middle,  and  bottom  of  the 
ranked  listing.   Based  on  this,  and  the  relative  ease  of 
collecting  data  on  each  of  the  factors,  the  following  factors 
were  tentatively  selected  for  evaluation  (numbers  refer  to 
ranks  in  Figure  2): 


Top 

1,    2,    3 

Middle 

14,    16,    17 

Bottom 

31,    32,    33, 

34 

Second,  a  factor  analysis  was  performed  on  the 
questionnaire  data  using  the  principal  components  method 
(see  Guilford,  1954;  Harman,  1965).   The  reason  for  this  was 
to  determine  if  items  on  the  questionnaire  represented  certain 
basic  dimensions  or  underlying  factors.   If  present,  such 
underlying  factors  could  then  be  used  as  a  guide  in  selecting 
individual  hypotheses  from  the  ranked  list  for  evaluation. 

The  factor  analysis  yielded  eleven  factors 
accounting  for  65.97.  of  the  variance.   However,  most  of  these 
factors  accounted  for  a  small  part  of  the  overall  variance, 
and/or  only  one  or  two  questionnaire  items  loaded  to  any 
appreciable  extent  on  a  given  factor.   The  factor  accounting 
for  the  largest  amount  of  the  variance  (9.4757.)  represented 
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essentially  those  items  that  were  ranked  high  on  the  question- 
naire (items  1,  2,  3,  5,  6,  and  9  in  Figure  2).   The  factor 
accounting  for  the  second  largest  amount  of  the  variance 
(9.4567.)  represented  essentially  those  questionnaire  items 
ranked  low  (items  17,  28,  29,  31,  32,  33  and  34  in  Figure  2). 
A  third  factor,  accounting  for  6.447.  of  the  variance,  had 
three  items  loading  very  high  on  it  (items  13,  15  and  21  in 
Figure  2),  and  represented  what  could  be  called  "project 
control".   The  other  eight  factors  were  judged  not  to  be 
meaningful  for  the  purposes  here,  either  because  they  accounted 
for  such  a  small  part  of  the  variance  or  because  only  one 
or  two  items  loaded  on  them  to  any  extent. 

It  was  apparent  from  the  factor  analysis  that  the 
high  ranked  and  low  ranked  items,  comprising  the  first  two 
factors,  were  well  represented  in  the  preliminary  list  of 
ten  hypotheses  selected  for  further  evaluation.   However, 
the  project  control  factor  was  not  represented  among  those 
ten.   Therefore,  item  13  from  the  ranked  list  (Figure  2)  was 
added  to  the  ten  hypotheses  previously  selected  for  further 
evaluation. 

Besides  the  above  eleven  hypotheses,  five  more 
were  selected  to  be  further  evaluated.   These  were  items  7, 
8,  12,  20,  and  21  from  the  ranked  list  of  hypotheses  (Figure  2) 
The  rationale  for  including  these  additional  hypotheses  was 
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(1)  ease  of  collecting  data  concerning  them;  merely  being 
in  an  organization  would  allow  getting  data  on  the  hypotheses 
at  virtually  no  extra  cost,  and  (2)  personal  interest  in  the 
hypotheses  because  of  related  research  currently  going  on 
or  because  of  emphasis  in  the  literature. 

As  was  noted  earlier,  space  was  provided  for 
each  respondent  to  write  in  any  factors  he  felt  important 
to  MIS  project  success.   All  write-in  responses  were  reviewed, 
but  no  new  hypothesis  was  deemed  necessary  from  this  review. 
In  most  cases,  the  respondents  merely  restated  in  a  different 
form  one  of  the  hypotheses  in  the  list  rated.   In  a  few  cases 
the  write-in  comments  were  not  closely  related  to  the 
hypotheses  rated;  however,  the  incidence  of  one  or  two  comments 
on  a  subject  was  not  viewed  as  justification  for  creating  a 
new  hypothesis. 

In  summary,  the  result  of  the  questionnaire 
analysis  was  the  selection  of  those  sixteen  hypotheses  identi- 
fied by  an  asterisk  (*)  in  Figure  2  to  be  evaluated  by  field 
study. 

Data  Collection  in  the  Field 
It  was  believed  that  the  best  means  of  collecting 
data  to  test  the  selected  hypotheses  was  through  interviewing 
key  persons  involved  in  developing,  or  using  the  products  of, 
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MIS  projects  in  several  organizations  in  several  industries. 
This  approach  was  to  comprise  a  "descriptive"  study,  using  the 
terms  of  Selltiz  et  al  (1967).   That  is,  the  emphasis  was  on 
finding  out  if  given  variables  were  associated,  in  this  case  the 
hypotheses  and  the  criteria  of  project  success. 

With  a  view  to  the  above  objectives,  a  questionnaire 
to  be  used  by  the  researcher  in  structured  interviews  was 
developed.   This  questionnaire  (see  Appendix  B)  was  aimed  at 
collecting  several  kinds  of  data  and  consisted  of  several  parts. 
First,  the  kinds  of  data  collected  with  the  questionnaire  can 
be  described  as  independent  variable  data,  dependent  variable 
data,  and  moderator  variable  data.   The  independent  variables 
were  the  hypotheses  selected,  as  outlined  above;  the  dependent 
variables  were  the  criteria  discussed  earlier;  and  the  moderator 
variables  represented  various  organizational  or  procedural 
factors  which  might  have  influenced  or  moderated  the  relation- 
ships among  the  independent  and  dependent  variables. 

The  parts  of  the  questionnaire  reflect  who  was  expected 
to  answer  certain  kinds  of  questions  about  a  project  in  the 
organization.   Thus,  certain  questions  about  the  organization 
itself  were  to  be  answered  by  the  manager  of  the  MIS  function. 
The  questions  specifically  related  to  the  project  were  to  be 
asked  of  the  project  leader,  and  the  questions  concerning  user 
perceptions  of  participation  and  satisfaction  with  the  products 
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of  the  project  were  to  be  directed  to  user  management 

.  1 
personnel. 

It  was  decided  that  twenty  MIS  projects,  drawn  from 
ten  separate  organizations,  would  constitute  an  adequate 
sample  for  the  type  of  research  to  be  conducted.   This  would 
represent  two  projects  in  each  of  the  ten  organizations.   It 
was  further  decided  that  of  the  two  projects  in  each  organiza- 
tion, one  of  them  should  be  viewed  as  relatively  successful, 
and  one  relatively  unsuccessful,  by  personnel  in  the  informa- 
tion systems  area  of  the  organization. 

The  general  guidelines  for  selecting  projects  for 
study  were  specified  as  follows: 

1.  Must  be  a  MIS  project  as  defined  in  Chapter  I; 
not  a  data  processing  application  with  no 

MIS  implications. 

2.  At  least  two  people  worked  full  time  in 
developing  the  project;  the  project  required 
at  least  three  elapsed  months  to  develop. 

3.  The  project  was  completed  and  implemented 
within  the  last  6-24  months. 

The  reason  for  guideline  2  above  was  to  preclude  studying 
very  small  projects  which  were  completed  by  one  person  over  a 
brief  time,  such  as  a  simple  report  generation  application  from 


The  following  references  were  used  in  developing  the 
questionnaire  for  conducting  interviews:   Cannell  &  Kahn, 
1953;  Edwards  &  Kenney,  1946;  Festinger  &  Katz,  1953;  Payne, 
1951;  Selltiz  et  al,  1967;  Torgerson,  1953;  Whitlock,  1963. 


-55- 

an  existing  file  of  data.   The  reasons  for  guideline  3  were 
to  avoid  projects  which  had  been  completed  for  so  long  that 
those  involved  were  not  available  for  interview,  or,  if 
available,  their  recollections  were  too  fuzzy  to  be  anywhere 
near  accurate;  and  to  avoid  projects  which  had  been  completed 
so  recently  that  there  was  inadequate  user  experience  with 
the  products  to  allow  fair  evaluation. 

Pretest  of  Interview  Instrument 

It  was  next  decided  that  a  pretest  of  the  questionnaire 
should  be  made  prior  to  actually  selecting  a  sample  and  making 
contacts  for  the  full  study.   The  objectives  of  the  pretest 
were  to  determine  if  data  of  the  types  wanted  could  be  gathered, 
and,  if  so,  were  the  questionnaire  and  associated  structured 
interview  effective  and  efficient  means  to  collect  these  data. 

Accordingly,  to  satisfy  these  objectives,  and  to 
obtain  estimates  of  how  long  the  data  collection  would  take 
for  each  organization  in  the  sample,  a  pretest  of  the  question- 
naire and  interview  technique  was  conducted  during  the  week  of 
4  May  1970,  in  a  Twin  Cities  financial  institution.   Based  on 
the  pretest,  it  was  concluded  that  the  data  collection  techniques 
planned  were  feasible,  and  would  provide  the  data  desired. 
However,  certain  very  minor  changes  to  the  questionnaire  appeared 
desirable  as  a  result  of  the  pretest  experience.   For  the  most 
part,  the  changes  merely  involved  revisions  of  wording  in 
questions  which  had  caused  some  confusion  among  pretest 
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respondents.   Two  questions  were  dropped  completely  from  the 
questionnaire  as  it  was  apparent  they  were  redundant.   These 
two  questions  elicited  essentially  the  same  information  from 
respondents  as  two  other  questions  which  were  left  in  the 
questionnaire.   Three  questions  were  added  to  the  questionnaire, 
and  two  were  modified  substantially,  in  order  that  project 
leaders  and  users  would  be  asked  identical  questions  concerning 
user  participation  and  implementation  problems.   None  of  these 
changes  altered  the  substance  of  the  questionnaire  to  any 
appreciable  extent,  and  were,  primarily,  corrections  for  over- 
sights brought  out  by  the  pretest.   After  revising  the  question- 
naire, work  began  on  selecting  the  organization  sample  to  be 
used  in  the  full  field  study. 

Organization  Sample  Selection 

As  was  stated  at  an  earlier  point,  it  was  desired 
that  the  ten  organizations  comprising  the  study  sample  represent 
different  industries  to  the  greatest  extent  possible.   This 
objective  for  selecting  an  organization  sample  was  constrained 
by  the  fact  that  it  was  also  desired  that  those  organizations 
in  the  sample  be  Associates  of  the  Management  Information  Systems 
Research  Center  (MISRC)  at  the  University  of  Minnesota. 
However,  this  constraint  was  not  a  very  serious  one  in  that  the 
Associate  firms  do  represent  a  wide  cross-section  of  industry 
types  in  the  Twin  Cities  area. 
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Wlth  a  view  to  the  above  requirements,  a  preliminary 
list  of  ten  firms  was  prepared.   Each  of  these  ten  firms  was 
then  sent  a  letter  from  the  Associate  Director  of  the  MISRC, 
indicating  the  nature  of  the  research  to  be  conducted,  the 
background  of  the  researcher,  and  a  request  that  the  firm 
participate  in  the  research  (Appendix  C).   Within  one  to 
two  weeks  from  the  time  the  letter  went  out,  the  researcher 
contacted  the  Associate  representative  of  each  firm  by  tele- 
phone to  request  an  appointment  for  the  purpose  of  discussing 
the  study  and  getting  agreement  from  the  firm  to  participate. 

During  the  last  two  weeks  of  July,  1970,  each  of 
the  ten  firms  originally  selected  was  visited  by  the  researcher, 
These  visits  generally  lasted  about  thirty  minutes,  during 
which  time  the  researcher  described  what  the  study  was  to 
consist  of,  the  kinds  of  data  to  be  collected,  and  the  types 
of  people  in  the  organization  who  would  be  involved  if  the 
firm  agreed  to  participate.   Additionally,  each  Associate 
representative  with  whom  the  researcher  visited  was  given  a 
brief  written  resume  of  the  proposed  research,  which  included 
the  estimated  duration  of  each  interview  to  be  conducted  and 
the  total  time  commitment  by  the  organization  (Appendix  D). 

Of  the  initial  ten  organizations  contacted,  all  but 
one  agreed  to  participate  in  the  study.   The  one  exception  was 
an  organization  which  had  begun  exploring  MIS  applications 
very  recently,  and  felt,  therefore,  that  there  would  not  be 
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any  projects  for  study  in  the  organization  which  staisfied  the 
researcher's  project  selection  specifications.   However,  to 
offset  this  organization's  not  participating  in  the  study,  one 
of  the  Associate  representatives  visited  desired  that  two 
separate  divisions  of  his  organization  be  included  in  the 
study.   The  researcher  agreed  to  this  after  satisfying  himself 
that,  for  all  practical  purposes,  the  two  highly  autonomous 
divisions  of  this  very  large  and  diversified  corporation  were 
separate  and  Independent  with  respect  to  the  factors  relevant 
to  the  study. 

Having  gotten  agreement  to  participate  in  the  study 
from  those  organizations  selected  for  the  sample,  a  schedule 
was  prepared  for  actual  data  collection.   This  schedule,  cover- 
ing the  period  from  early  October  to  mid-December,  1970,  allowed 
at  least  three  days  for  collecting  data  in  each  organization. 
Bach  organization  was  again  contacted  by  telephone  to  arrange 
the  dates  for  interviewing  in  the  organization.   The  only 
appointment  made  at  this  time  was  with  the  director  of  the 
information  systems  function  or  other  individual(s)  whom  had 
been  designated  by  the  Associate  representative  to  coordinate 
data  collection  in  the  organization. 

Data  Collection  Procedures 

The  data  collection  procedures  followed  in  each 
organization  which  participated  in  the  study  were  as  follows: 


-59- 


1.   An  initial  meeting  with  the  manager  of  the 
information  systems  function  and/or  his 
designated  representatives  was  opened  by 
explaining  precisely  what  data  were  to 
be  collected,  and  from  whom,  after  a 
general  description  of  the  research  study 
had  been  provided.   Next,  the  organization 
representatives  were  asked  to  suggest 
projects  for  study  which  met  the  project 
selection  specifications  described  earlier 
in  this  chapter.   This  proved  to  be  a  more 
time  consuming  task  than  expected,  and,  in 
some  cases ,  required  as  long  as  three  hours 
rather  than  the  thirty  minutes  estimated 
from  the  pretest  experience.   Those  two 
projects  which  best  satisfied  the  selection 
specifications  were  chosen  for  study.   The 
organization  representatives  were  then 
requested  by  the  researcher  to  arrange 
appointments  with  the  project  leaders  of 
the  two  projects  selected.   Finally,  the 
manager  of  information  systems,  or  his  repre- 
sentative, was  asked  to  complete  pages  1-3  of 
the  questionnaire.   This  was  the  only  part  of 
the  questionnaire  which  was  not  completed  during 
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interviews  with  the  researcher.   However,  on 
several  occasions  the  researcher  did  assist 
the  respondents  for  pages  1-3  by  clarifying 
points  that  were  confusing,  or  in  making  certain 
that  the  respondents  interpreted  the  questions 
as  the  researcher  intended. 
2.   The  meeting  with  the  project  leader  was  opened 
with  a  brief  description  of  the  nature  of  the 
study  and  the  data  to  be  collected,  and  with 
assurances  to  the  project  leader  that  all  of 
his  responses  would  be  held  in  confidence  by 
the  researcher.   The  remainder  of  the  1%  to  3 
hours  spent  with  the  project  leader  was   devoted 
to  completing  the  appropriate  portions  of  the 
questionnaire.   The  respondent  was  given  a 
copy  of  the  questionnaire  to  read  as  the 
researcher  asked  the  questions. 
Upon  completion  of  his  portion  of  the  question- 
naire, the  project  leader  was  requested  to 
arrange  appointments  with  at  least  two  managers 
in  the  area(s)  which  received  the  products  of 
the  project.   For  this  purpose,  a  staff  analyst 
or  similar  person  was  considered  to  be  a  manager. 
In  general,  if  a  user  was  involved  in  making 
planning  and/or  control  decisions  in  a  functional 
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area  of  responsibility,  he  was  considered  a 
manager  whether  he  had  any  subordinates  or  not. 
Finally,  the  project  leader  was  requested  to 
arrange  an  appointment  with  the  computer  opera- 
tions manager,  or  other  knowledgeable  individual 
in  the  computer  operations  function  who  could 
answer  those  questions  pertaining  to  computer 
operations  for  the  project. 

3.  The  meeting  with  each  user  was  opened  in  exactly 
the  same  manner  as  the  meeting  with  the  project 
leader.   After  the  preliminary  comments  by  the 
researcher  about  the  study,  the  remainder  of 

the  fifteen  to  thirty  minutes  spent  with  the 
user  was   devoted  to  the  completion  of  the 
relevant  portions  of  the  questionnaire.   As 
with  the  project  leader,  the  user  was  provided 
a  blank  questionnaire  to  read  as  the  researcher 
asked  the  questions. 

4.  The  same  preliminary  explanations  on  the  nature 
of  the  study,  and  so  forth,  were  provided  to 
the  computer  operations  management.   In  most 
cases,  the  computer  operations  data  for  both 
projects  in  an  organization  were  collected  in 
one  interview.   This  interview  generally  lasted 
from  ten  to  fifteen  minutes. 
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Steps  2  and  3  of  the  above  procedure  were  repeated  for  the 
second  project  studied  in  the  organization. 

Data  Analysis  Procedures 

After  all  the  data  had  been  collected  in  all  ten 
organizations  in  the  sample,  the  interview  forms  were  reviewed 
and  coded  by  the  researcher.   Since  the  questionnaire  was 
originally  developed  to  facilitate  coding  and  conversion  of 
the  data  to  punched  card  form,  this  was  a  relatively  easy  task. 

Several  methods  of  data  analysis  were  considered, 
but  it  was  decided  that  Kendall's  (1962)  tau  statistic  provided 
the  best  measure  available  of  relationships  among  the  variables 
for  the  following  reasons: 

1.  The  tau  statistic  is  robust  and  relatively 
powerful  in  dealing  with  ranked  data  contain- 
ing ties  among  ranks. 

2.  Being  a  measure  of  relationship  among  ordinal 
variables,  there  are  no  assumptions  concerning 
distribution  of  the  raw  data  for  the  tau  statistic 

3.  The  S  statistic,  which  is  actually  the  numerator 
of  the  tau  statistic,  and  is  a  measure  of  the 
agreement  or  disagreement  between  a  pair  of 
rankings,  is  distributed  approximately  normally 


For  discussions  of  the  tau  statistic,  its  distribution 
under  various  conditions,  and  corrections  for  continuity,  see 
Burr,  1960;  Goodman  and  Kruskal,  1954;  Hoffding,  1947;  Kendall, 
1947;  1962;  Sillitto,  1947;  Somers ,  1962;  and  Whitfield,  1947. 
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for  N  ^  9.   The  S  statistic  can,  therefore, 
be  converted  to  a  normal  deviate, Z(s) ,  after 
computing  the  variance  of  S.   This  normal 
deviate  can  then  be  tested,  using  the  normal 
probability  table,  to  ascertain  the  signi- 
ficance level  of  the  relationship  between 
two  rankings. 
4.   since  the  data  collected  could  be  arranged 
in  naturally  ordered  bivariate  tables,  the 
computation  of  tau,  S,  and  Z(s)  could  be 
accomplished  quite  easily  using  the  UMST 
(1969)  programs  available  for  the  University 
of  Minnesota's  CDC  6600  computer  system. 
The  first  step  taken  in  analyzing  the  data  collected 
was  to  prepare  a  descriptive  profile  of  the  sample.   This 
descriptive  profile  was  broken  into  two  parts:   descriptive 
statistics  on  the  organizations  in  the  sample,  and  descriptive 
statistics  on  the  projects.   These  descriptive  statistics 
consisted  of  means,  medians,  and  ranges  on  those  variables  which 
were  amenable  to  such  treatment,  and  distributions  of  scores 
on  the  variables  which  were  not.   These  descriptive  statistics 
on  the  organizations  and  the  projects  are  presented  in  Chapter  5. 

The  second  step  in  analyzing  the  data  was  to  compute 
the  tau  statistics  for  relationships  among  the  variable  rankings. 
This  step,  in  turn,  was  divided  into  three  substeps. 
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First,  it  was  necessary  to  determine  whether  or  not 
those  variables  which  were  originally  intended  to  be  composites 
of  several  separate  items  were,  in  fact,  valid  composites  repre- 
senting the  same  factor,  or  were,  in  reality,  separate  factors 
which  should  not  be  aggregated.   The  variables  in  question 
were  User  Participation-PL  (Var.  41),  User  Participation-User 
(Var.  46),  User  Satisfaction-User  (Var.  54),  project  leader's 
perception  of  success,  and  operations  success.   To  make  this 
determination,  Pearson  product -moment  correlation  coefficients 
were  computed  for  all  of  those  separate  items  comprising  a 
supposed  aggregate  factor.   This  computation  was  accomplished 
with  computer  program  UMST  530.   If  any  item  within  a  supposed 
composite  factor  was  not  correlated  at  least  .50  with  every 
other  item  in  that  composite  factor,  that  item  was  considered 
to  be  a  separate  measure  and  dropped  from  the  composite  factor. 
It  was  found  that  Variables  41,  46,  and  54  did,  in  fact,  have 
items  which  were  intercorrelated  at  least  .50,  and,  therefore, 
were  acceptable  as  composite  measures  of  the  factors  in 
question.   However,  the  intercorrelations  of  separate  items 
in  the  assumed  composite  variables  representing  the  project 
leader's  perception  of  success  and  computer  operations  success 
were  so  low  as  to  preclude  their  being  considered  composite 
factors.   Consequently,  computer  operations  success  and  the 
project  leader's  perception  of  success  were  treated  in  all 
further  analysis  as  two  separate  variables  (59  and  60),  and 
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four  separate  variables  (55-58),  respectively.   (See  Appendix  E 
for  descriptions  of  all  variables  used  and  their  scoring.) 

Next,  in  order  to  determine  roughly  which  variable 
rankings  were  related  enough  to  warrant  further  investigation 
of  the  relationships,  UMST  540  was  used  to  provide  a  tau 
statistic  for  every  possible  cross-classification  among  the 
61  variables  in  the  study.   The  tau  statistic  computed  by 
UMST  540  was  based  on  Slegel's  (1956)  formula  for  tau-B,  and 
gave  an  inflated  normal  deviate  where  there  were  ties  present 
in  the  rankings.   This  inflated  normal  deviate  resulted  from 
not  making  any  correction  for  continuity  prior  to  computing 
the  normal  deviate.   However,  the  computation  with  UMST  540  was 
relatively  simple  and  fast,  and  provided  an  excellent  means  for 
identifying  those  variable  relationships  which  should  be  explored 
further  in  the  last  substep. 

Finally,  UMST  590  was  used  to  derive  tau,  S,  and 
Z(s)  statistics  for  all  criterion  variables  cross-classified 
against  each  other,  all  hypothesis  variables  cross-classified 
against  all  criterion  variables,  and  all  other  relationships 
which  were  indicated  as  possibly  significant  from  the  previous 
run  based  on  Slegel's  formula  for  tau-B.   UMST  590  computations 
are  based  on  Kendall's  (1962)  formulas  for  tau-B,  tau-C,  S, 
the  variance  of  S,  and  the  normal  deviate  of  S.   (See  Appendix 
F  for  these  computing  formulas.) 


PART  III 
RESULTS  OF  THE  STUDY 

Part  III  contains  the  substance  of  the  research 
study.   It  Is  divided  into  three  chapters.   Chapter  V 
presents  descriptive  data  on  the  organizations  in  the  sample 
and  on  the  projects  studied.   These  descriptive  data  consist 
of  means ,  medians,  and  ranges  where  appropriate,  and  of 
distributions  of  responses  by  category  where  they  are 
appropriate. 

Chapters  VI  and  VII  present  the  results  of  the 
statistical  analysis  of  the  sample  data.   The  primary  means 
of  analysis  was,  as  discussed  in  Chapter  IV,  the  determination 
of  association  among  variables  using  Kendall's  rank  correlation 
statistic,  tau.   The  tables  of  association  are  Included  with 
the  text  to  facilitate  reader  reference. 
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CHAPTER  V 
DESCRIPTION  OF  THE  STUDY  SAMPLE 

Organizations  in  the  Sample 

The  ten  organizations  which  participated  in  the  study 
were  all  located  in  the  Twin  Cities  area.   Two  of  the  organi- 
zations were  highly  autonomous  divisions  of  a  large,  multi- 
business  corporation.   The  remaining  eight  organizations  were 
completely  unrelated. 

The  ten  organizations  were  categorized  into  the 
following  industry  groupings: 

1.  Manufacturing „  .5 

a.  Consumer  products  using 
continuous  process 

production  technology „  .  .3 

b.  Industrial  products  using 
assembly  line/mass  production 
technology 2 

2.  Non-manufacturing 5 

a.  Wholesale,  retail  trade 

b.  Transportation  

c.  Finance.  ...  

d.  Utility 


e.   Commodity  merchandising 
and  processing  .  .  .  .  , 


Measures  of  Organization  Size 

The  organization  sample  represented  considerable  range 
in  terms  of  relative  organization  size.   This  fact  is  reflected 
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in  Table  1,  which  shows  the  mean,  median,  and  range  for  four 
measures  of  organization  size. 

TABLE  1 
Measures  of  Organization  Size 

Mean Median Range 

Assets  (Var.  1)  thousands     $593,286   $170,000   $   60,000- 
(7  organizations  only)  $2,875,000 

Sales  (Var.  2)   thousands     $647,000   $491,000   $    76,000- 
(8  organizations  only)  $2,600,000 

Employees  (Var.  3)  9,931      6,100       560- 

47,900 

Customers  (Var.  4)  147,693     27,500       500- 

(8  organizations  only)  450,000 

Throughout  Part  III,  the  variable  number  is  presented  whenever 
a  variable  is  discussed  or  entered  in  a  table.   This  has  been 
done  to  facilitate  reader  reference  to  Appendix  E  where  all  of 
the  variables  in  the  study  are  fully  described.   No  particular 
significance  should  be  attached  to  the  variable  numbers  them- 
selves, however.   The  numbers  were  assigned  by  the  researcher 
in  essentially  the  order  the  variables  appeared  in  the  study 
questionnaire  (Appendix  B). 


Other  Relevant  Organization  Attributes 

Several  other  attributes  of  the  organization  sample 
besides  organization  size  are  relevant  to  the  present  study. 
Included  in  this  category  are  the  reported  degree  of  organiza- 
tional change  over  the  last  three  years;  the  reported  degree  of 
centralization  of  the  organization's  MIS  activities;  the 
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organizational    location  of    the  manager  of    the    information 
systems    function;    the  organization's   hardware    investment  and 
costs;    and    the  size  of   the  systems   and  programming   staff.      The 
data   for   all  of    these  attributes   are  shown   in  Tables    2-4. 

TABLE    2 

Organization  Change,  Centralization,  and 
Level  of  the  Information  Systems  Manager 

Number  of 
Organizational  Attributes Organizations 

•  Organization  Change  over  Last  Three  Years  (Var.5): 

Little  or  no  change 5 

Minor  changes 1 

Considerable  change  4 

•  Centralization  of  Organization  MIS  Activities 
(Var.  8): 

All  design,  analysis,  and  programming 
performed  by  corporate  MIS  staff 
regardless  of  origin  of  project.  ...        8 

Exactly  as  above,  except  that  small  opera- 
tions research  group  developed  some 
of  its  own  projects 2 

•  Level  of  Information  Systems  Manager  (Var.  9): 

Number  of  hierarchical  levels  between 
manager  directly  responsible  for 
Information  systems  function  and 
the  top  operating  executive: 

0  Levels    3 

1  Level      5 

2  Levels    2 

•  Immediate   Superior  of  Manager  of   Information 
Systems: 

Top  operating   executive    3 

Administrative  vice-president    „    .  3 

(or   similar   position) 

Controller 3 

Market  research   director 1 


TABLE  3 
Hardware  Investment  and  Costs 
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Mean        Median Range 


Hardware    Investment/Sales    (Var.    6)  .947.  .907.        .107.-2.17. 

Hardware  Expense/Budget   (Var.    7)  357.  357.  137.-507. 


TABLE  4 
Size  of  Systems   and   Programming  Staff 


Mean        Median Range 


Number   in  Systems   and   Programming  50  35  7-156 

(Var.    10) 

Systems   Staff/Total   Employees  1.317.        .527.  .257.-8.27. 

(Var.    61) 
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Projects  in  the  Sample 
This  section  of  Chapter  V  deals  with  the  descriptive 
findings  relative  to  the  twenty  projects  themselves.   The 
variables  to  be  discussed  include  those  pertaining  to  project 
scope  and  size,  project  personnel,  project  procedural  features, 
and  project  success. 

Before  taking  up  the  specific  variables  for  which 
data  were  collected,  several  general  points  concerning  the 
project  sample  are  in  order.   First,  it  is  reiterated  that  only 
projects  which  met  the  criterion  of  being  MIS  projects  were 
included  in  the  sample.   Projects  which  were  merely  data 
processing  applications  with  no  management  information  spinoffs 
were  rejected.   In  some  instances  the  satisfaction  of  this 
criterion  severely  narrowed  the  range  of  projects  available  for 
study.   In  fact,  in  some  organizations  it  was  only  after  consider- 
able probing  that  valid  MIS  projects  could  be  identified.   This 
situation  led  to  accepting  projects  for  study  which  did  not  fall 
within  the  desired  time  and  size  parameters  given  in  Chapter  IV. 
Specifically,  the  following  deviations  were  made  from  the 
criteria  for  project  selection: 

1.   It  was  desired  that  all  projects  in  the  sample 
had  had  at  least  two  people  who  worked  primarily 
on  the  project  during  its  active  development 
periods.   Eighteen  of  the  projects  met  this 
requirement,  but  two  did  not.   Two  projects 
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were  developed  entirely  by  one  person. 
However,  they  were  of  such  scope  that  they 
were  considered  to  be  valid  components  of 
the  sample. 
2.   It  was  desired  that  all  projects  had  been 
completed  between  six  and  twenty-four 
months  prior  to  the  time  of  the  study. 
This  proved  to  be  the  most  difficult 
requirement  to  satisfy,  in  that  only 
fourteen  projects  were  completed  between 
November,  1968,  and  July,  1970.   Three 
projects  were  completed  prior  to  November, 
1968,  the  earliest  of  which  was  implemented 
in  September,  1967.   This  means  that  a  little 
more  than  three  years  had  elapsed  between 
the  time  the  project  was  completed  and  the 
time  the  data  were  collected.   While  the 
period  of  recollection  was  thus  over  a 
year  longer  than  the  maximum  recollection 
period  desired,  it  was  impossible  to 
estimate  how  much  the  data  were  biased  by 
increased  forgetting  of  the  specifics  of 
the  project.   This  same  observation  holds 
for  all  of  the  projects  beyond  the  twenty- 
four  month  limit,  and,  Indeed,  for  all  of 
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the  projects  in  the  sample,  excepting  the 
very  recent  ones. 

Three  projects  were  completed  after  July, 
1970,  the  most  recent  of  which  was  imple- 
mented in  October,  1970.   The  primary  consid- 
eration with  such  recent  projects  was,  of 
course,  whether  or  not  the  users  had  had 
adequate  time  to  evaluate  the  results  of 
the  projects.   The  researcher  was  satisfied 
that  two  of  the  projects  completed  since  July, 
1970,  did  have  an  adequate  experience  base 
for  fair  evaluation.   One  of  these  projects, 
completed  in  September,  resulted  in  a  daily 
output  which  had  provided  ample  opportunity 
for  shakedown  and  appraisal.   The  other 
project,  completed  in  August,  had  been 
evolving  over  a  period  of  twelve  months, 
with  very  close  interplay  between  the  users 
and  the  systems  staff.   In  effect,  this 
project  was  not  considered  completed  until 
the  users  had  indicated  satisfaction  with 
the  products  after  several  months  of  evaluat- 
ing alternative  outputs.   The  third,  and  most 
recent  project,  completed  in  October,  1970, 
was  studied  in  early  December.   To  check  on 
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the  users'  perceptions  of  the  project  after 
two  additional  months  experience,  a  follow-up 
was  made  with  the  users  in  early  February, 
1971.   Although  still  two  months  short  of 
the  desired  six  months  minimum,  there  is  no 
reason  to  suspect  bias  enough  to  warrant  not 
including  the  project  in  the  sample. 

General  Types  of  Projects 

As  previously  stated,  all  of  the  projects  studied 
were  management  information  system  projects;  that  is,  outputs 
were  generated  for  a  manager  at  some  level  which  were  inputs 
to  his  dec is  ion "making  process.   This  general  classification 
between  MIS  projects  and  data  processing  projects  was  further 
broken  down  into  the  subcategory  of  best  fit.   The  subcategories 
and  numbers  of  projects  in  each  are  shown  in  Table  5. 

Origin  of  Projects  and  Nature  of  Objectives 

Of  the  twenty  projects  studied,  657.  were  initiated 
by  user  managements.   This  would  seem  to  indicate  a  desire  on 
the  part  of  functional  managers  to  utilize  the  computer  as  an 
aid  in  carrying  out  their  dec is ion -making  responsibilities. 
However,  the  reported  objectives  in  initiating  projects  did 
not  always  seem  to  be  as  clear  and  specific  as  they  might 
have  been.   These  observations  are  based  on  the  distributions 
of  project  origins  and  project  objectives  shown  in  Table  6. 
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TABLE  5 
Types  of  MIS  Projects  Studied 


Number  of 
Projects 
Types  of  MIS  Projects of  Type 


Models:   projects  involving  management 
science  techniques  such  as  simulation, 
mathematical  programming,  or  forecasting; 
generally  projects  aimed  at  providing 
inputs  to  planning  processes  


2.  Data  processing  spinoffs:   projects 
which  evolved  from  operational  control 
systems  at  the  lower  levels  of  the 
organization;  generally  spinoffs  from 
accounting  and  logistics  systems,  and 
aimed  at  providing  input  to  control 
processes  through  monitoring,  triggered, 
or  demand  reports  (see  B lumen thai,  1969, 
p.  51) 

3.  Data  collection  and  analysis:   projects 
which  were  developed  from  the  ground  up 
with  specific  uses  and  requirements; 
generally  involved  setting  up  means  to 
collect  desired  information,  creating 
the  necessary  data  base,  and  developing 
analytical  routines  to  manipulate  the 
data  base;  these  projects  aimed  at 
providing  inputs  to  both  planning  and 
control  processes;  an  example  of  such  a 
project  would  be  a  marketing  intelligence 
s  ys  tern 
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TABLE  6 
Project  Origins  and  Objectives 


Number  of 
Projects 


Origin  of  Project  (Var.  11) 

User 13 

Top  level  management 3 

Information  systems  staff  4 

Project  Objectives  (Var.  12) 

Specific,  measurable,  written  objectives.  .  .  5 

Specific,  non-measurable,  written  objectives.  6 

General,  clear,  written  objectives 2 

General,  unwritten  objectives  3 

Rather  vague  objectives  4 


Project  Scope  and  Size 

The  following  variables  relate  to  the  size  and  scope 

of  the  projects  in  the  sample: 
Complexity  -  (Var.  13) 

There  was  no  objective  measure  of  comparative 
complexity  of  projects  in  the  sample.   The 
ratings  of  project  complexity  were  entirely 
relative  to  the  individual  project  leaders  and 
the  systems  environments  in  their  organizations. 
For  this  reason,  the  distribution  given  below  is 
not  the  distribution  of  complexity  of  all  sample 
projects  based  on  some  objective  scale  applied 
to  all  projects;  rather,  it  is  a  distribution  of 
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the  ratings  of  complexity  assigned  by  each 
project  leader  relative  to  his  own  experience: 

Highly  complex 3 

Above  average  complexity  .  .  .5 

Average  complexity 8 

Below  average  complexity  .  .  .2 

Very  low  complexity 2 

Size  and  composition  of  project  staffs 

Table  7  provides  a  breakdown  of  the  composition  and 

size  of  project  staffs  for  the  projects  in  the  sample: 

TABLE  7 
Composition  of  Project  Staffs 


Mean   Median   Range 


Number  on  Project  (Var.  17) 
Number  of  Analysts  (Var.  18) 
Number  of  Programmers  (Var.  19) 
Number  of  Users  -  N=13  (Var.  20) 


5.0  4  1-12 

1.5  1  1-3 

2.5  2  1-8 

1.8  2  1-3 


Outside  consultants  used  on  three  projects 


Time  spent  on  projects 

Table  8  contains  data  on  the  elapsed  time  and  the  man 

months  spent  on  the  projects  in  the  sample: 
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TABLE  8 
Time  Spent  on  Projects 


Mean   Median   Range 


Elapsed  Months  (Var.  21)  12.1    10.5     2-48 

Man  Months  (Var.  22)  22.0    10.0     4-87 

Attributes  of  Project  Personnel 

The  following  variables  relate  to  the  education, 
systems  experience,  and  organization  experience  of  those  who 
worked  directly  on  the  projects  in  the  sample: 

Systems  experience 

Table  9  contains  data  on  four  measures  of  the 

systems  experience  of  those  who  worked  directly 

on  the  projects  studied: 

TABLE  9 
Systems  Experience  of  Project  Personnel 


Mean   Median   Range 


Systems  Experience  of  Project  Leader-- 

months  (Var.  24)      66.2    43.5    0-300 

Mean  Systems   Experience  of   Project 
Personnel    Including  Users-- 
months  (Var.    25)  47.3  28.5  8-180 

Mean  Systems   Experience  of  Project 

Personnel   from  MIS   Staff  Only— 

months  (Var.    26)  52.5  37.5        11-180 

Two  or  More  Years   Systems   Experience-- 

proportion  (Var.    27)  587.  50%  0-1007. 
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It  should  be  noted  that  the  values  given  in  Table  9 

for  Variables  25,  26,  and  27  are  not  those  for  all 

project  personnel  in  the  sample.   Rather,  they 

are  the  values  derived  from  the  means  or  proportions 

that  Variables  25,  26,  and  27  represented  for  each 

project.   Thus,  47.3  months  is  the  mean  of  twenty 

project  means,  not  the  mean  systems  experience  for 

all  individuals  who  worked  on  projects  in  the  sample. 

Impact  of  systems  experience  -  (Var.  28) 

With  respect  to  the  impact  of  systems  experience  on 

project  success,  project  leaders  generally  indicated 

that  such  experience  was  a  contribution  to  project 

success,  and  lack  of  systems  experience  hindered 

success  somewhat.   However,  as  the  following 

distribution  of  Variable  28  shows,  257.  of  the 

project  leaders  felt  that  prior  systems  experience 

was  of  no  importance  one  way  or  the  other  to 

project  success: 

Experience  critical  to  success 4 

Experience  contributed  somewhat 

to  success 8 

Experience  of  no   importance 5 

Lack  of  experience  of  some  staff 

members   detrimental 3 

Education 

Table  10  contains  data  on  three  measures  of  the 

educational  levels  of  those  who  worked  directly 
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on  the  projects  studied: 

TABLE  10 
Formal  Education  of  Project  Personnel 


Mean   Median   Range 


College  Degree  - 

proportion  (Var.  30)       65%     757.    0-1007. 

Two  Years  College  - 

proportion  (Var.  31)       797.     877.   33-1007. 

Mean  Years  Formal  Education 

(Var.  32)      15.3    15.3    13-18 


As  with  systems  experience,  it  should  be  noted  that 
the  above  values  are  derived  from  project  statistics, 
not  individual  measures. 
Organization  experience 

Table  11  contains  data  on  two  measures  of  the 
experience  project  personnel  had  in  their  respect- 
ive organizations  at  the  beginning  of  the  projects 
studied. 

TABLE  11 
Organization  Experience  of  Project  Personnel 

Mean   Median   Range 

Mean  Years  Organization 

Experience  (Var. 33)  6.75    6.75   1.5-20.0 

Two  or  More  Years  Organization 

Experience  -  proportion    (Var. 34)   697.     697.   25-1007. 
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As  with  systems  experience  and  education,  the  above 
values  are  derived  from  project  statistics,  not 
individual  measures. 

Turnover  -  (Var.  29) 

Of  the  twenty  projects  in  the  sample,  only  three 
experienced  any  turnover  among  the  project  staff.   Two  of 
these  projects  had  a  turnover  rate  of  50%,  while  the  third 
project's  turnover  rate  was  677..   For  the  three  projects 
which  experienced  staff  turnover,  the  project  leader's 
appraisal  of  the  impact  of  this  turnover  on  project  success 
was  distributed  as  follows: 

Very  detrimental 1 

Somewhat  detrimental 1 

Contributed  to  success  ...  .1 

Procedural  and  Organizational  Attributes  of  Projects 

Tables  12  and  13  contain  breakdowns  of  the  study 
sample  according  to  several  organizational  and  procedural 
attributes  of  the  projects.   In  addition  to  the  information  in 
these  tables,  several  amplifying  or  explanatory  comments 
concerning  the  attributes  shown  are  in  order. 

As  can  be  seen  from  Table  12,  thirteen  of  the  projects 
studied  (65%)  were  developed  by  project  teams  comprised  of  user 
representatives  working  with  information  systems  staffs.   Of 
those  thirteen,  two  project  teams  had  user  representatives 
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studied  (65%)  were  developed  by  project  teams  comprised  of  user 
representatives  working  with  information  systems  staffs.   Of 
those  thirteen,  two  project  teams  had  user  representatives 
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assigned  full  time  during  the  analysis  and  design  phases;  the 
remaining  eleven  project  teams  had  user  representatives  working 
on  the  projects  on  a  part-time  basis.   Further,  for  eleven  of 
the  thirteen  projects  where  a  team  was  utilized,  the  user 
representatives  who  participated  on  the  teams  were  also  respon- 
sible for  implementing  the  completed  projects  in  their  respective 
departments,  and  for  maintaining  liaison  with  the  information 
systems  staffs  after  implementation.   For  the  other  two 
projects,  team  participants  during  development  were  not  the 
ones  responsible  for  implementation  and  continuing  liaison. 

For  the  thirteen  projects  which  were  developed  by 
teams,  the  mean  value  for  the  proportion  of  total  project  man- 
months  accounted  for  by  user  personnel  (Var.  15)  was  187.;  the 
median  value  was  18.57.;  and  the  range  was  4-517.. 

Finally,  project  leaders  generally  felt  that  user 
representatives  on  project  teams  made  valuable  contributions 
to  project  success.   Nine  of  the  project  leaders  who  headed 
teams  appraised  user  member  contributions  as  very  great,  as 
critical  to  project  success.   The  other  four  project  leaders 
felt  that  user  members  made  some  contributions,  and  that  the 
projects  would  not  have  turned  out  as  well  as  they  did  without 
the  users. 

With  respect  to  project  documentation,  formal  standards 
for  documentation  were  considered  to  exist  even  if  such 
standards  were  applicable  to  only  the  programming  phase  of  a 
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project.   It  should  also  be  pointed  out  that  the  quality  of 
project  documentation  was  not  measured  against  any  universal 
objective  standard  or  scale.   Rather,  the  documentation  quality 
distribution  shown  in  Table  12  reflects  the  appraisals  of 
Individual  project  leaders  as  to  how  good  they  thought  the 
project  documentation  was  for  their  respective  projects. 

The  degree  of  user  participation  in  the  analysis 
and  design  of  each  project  was  appraised  by  both  user  personnel 
and  the  project  leader.   The  amount  of  agreement  between  those 
appraisals  will  be  taken  up  in  a  subsequent  chapter.   However, 
it  is  worth  noting  at  this  point  that  there  was  a  tendency 
for  fairly  high  levels  of  user  participation  in  the  projects 
studied,  whether  viewed  from  the  users'  side  or  the  project 
leader's  perspective.   This  observation  is  supported  by  the 
statistics  in  Table  13. 

TABLE  12 
Project  Procedural  and  Organizational  Attributes 


Number  of 
Projects 


Project  Team  (Var.  14): 

Yes 13 

No 7 

Combination  Analyst/ Programmer  (Var.  23): 

Combination  9 

Separate 11 

Documentation  Standards  (Var.  35): 

Yes 16 

No 4 
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TABLE  12  (Cont.) 
Project  Procedural  and  Organizational  Attributes 


TABLE  13 
User  Participation 


Number  of 
Projects 


Quality  of  Documentation  (Var.  36): 

High  quality 6 

Adequate 8 

Somewhat  Inadequate  ....  5 

Low  quality 1 

Project  Control  (Var.  42): 

Formal  or  informal  report  of  progress 

against  plan  required  of  project 

staff  at  least  monthly 11 

No  regular  progress  reporting 

required  of  project  staff 9 

Programming  Language  (Var.  43): 

COBOL 15 

FORTRAN  2 

Assembly 3 

Generalized  Software  Problems  (Var.  44): 

No  problems 15 

Minor  problems 4 

Major  problems 1 


Mean   Median   Range 


User  Participation -PL        (Var.  41) 

possible  range:   3-15  11.5 


User  Participation-User  (Var.   46) 

possible  range:      40-200  153 


12.5  4-15 


162  83-200 
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Project  Success 

It  was  stated  in  Chapter  III  that  four  criteria  of 
project  success  were  defined  for  the  study:   time,  cost,  user 
satisfaction,  and  computer  operations  problems.   The  performance 
of  the  projects  on  each  of  these  criteria  is  shown  in  Table  14. 
In  addition  to  what  is  shown  in  the  table,  the  following  comments 
concerning  the  criteria  are  relevant: 
Time 

Of  the  twenty  projects,  six  were  completed  within 
the  times  estimated;  the  remaining  fourteen 
exceeded  their  time  estimates  by  varying  amounts 
up  to  900%.   Of  the  six  projects  which  were 
completed  on  time,  overtime  was  required  to  do  so 
for  three  of  them.   One  of  the  three  on-time 
projects  which  had  considerable  overtime  also 
had  substantial  programmer  resources  added  to 
the  staff  to  meet  the  time  deadline. 
Cost 

An  interesting  finding  regarding  project  cost 
was  that  nine  of  the  twenty  projects  in  the 
sample  had  no  cost  budget.   No  cost  estimates 
were  ever  made,  or  at  least  ever  recorded  or 
referred  to  later,  for  nearly  half  of  the 
projects  studied.   Since  it  was  impossible  to 
determine  if  a  project  was  completed  within  its 
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cost  budget  If  such  a  budget  never  existed, 
the  cost  data  for  these  projects  were  estimated, 
based  on  man -months  data  and  estimates  of 
computer  test  time.   This  seemed  to  be  a 
reasonable  approximation  to  project  cost, 
since  by  far  the  largest  part  of  project  cost 
was  accounted  for  by  manpower  cost,  which, 
in  turn,  was  a  direct  function  of  man -months 
spent  on  a  project.  Where  program  test  time 
was  determined,  through  discussions  with  the 
project  leader,  to  be  of  any  consequence  in 
the  total  project  cost,  a  factor  for  computer 
cost  was  included  in  the  cost  estimates. 
Using,  then,  the  best  estimates  that  could  be 
devised  for  project  cost  data,  it  was  found 
that  only  two  of  twenty  projects  were  completed 
within  their  cost  budgets.   The  remaining 
eighteen  projects  exceeded  their  cost  estimates 
by  varying  amounts  up  to  500%. 
User  satisfaction 

Since  five  separate  factors  were  aggregated 
together  to  form  the  measure  of  user  satisfac- 
tion (Var.  54),  the  range  of  possible  values 
for  this  measure  was  50-250,  with  50  represent- 
ing extremely  low  user  satisfaction  and  250 
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representing  extremely  high  user  satisfaction. 
As  the  statistics  in  Table  14  indicate,  a 
fairly  high  level  of  user  satisfaction  existed 
for  most  projects  in  the  sample.   If  the  sum 
of  the  middle  of  the  five  possible  ratings  on 
the  scale  for  each  of  the  five  factors  was 
taken  as  an  "average"  level  of  user  satisfaction, 
the  resulting  "average"  value  would  be  150. 
It  can  be  seen  that  both  measures  of  central 
tendency  shown  in  Table  14  for  the  actual 
sample  well  exceeded  that  hypothetical  value. 
Computer  operations  problems 
In  general,  the  projects  in  the  sample  were 
successful  in  terms  of  not  creating  problems 
in  the  computer  operations  function.  With  a 
five  point  scale,  ranging  from  very  serious 
problems  at  the  low  end  to  no  problems  at  the 
high  end,  a  hypothetical  average  would  have 
fallen  in  the  middle  of  the  scale,  or  at  a 
value  of  3,  which  represents  moderate  problems 
for  computer  operations.   However,  the  mean 
value  for  computer  operations  problems  (Var.  59) 
in  the  sample  was  3.95,  and  there  was  no 
project  in  the  sample  rated  below  3  on  the  scale. 
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Th  e  distribution  of  the  ratings  on  this 

criterion  was  as  follows: 

No  problems 5 

Very  minor  problems.    .    .    9 
Moderate  problems.    ...    6 

TABLE    14 
Project  Success 


Mean       Median       Range 


Actual   Time/Estimated  Time  (Var.  48)  209.6%   139.57.  75-9007. 

Actual   Cost/Estimated  Cost  (Var.  51)  194.77.   151.57.  82-5007. 

User  Satisfaction-User  (Var.  54)  175.8      192.5  80-220 
(possible  range:      50-250) 

Computer  Operations   Problems  (Var.  59)  3.95  4.0  3-5 

(possible  range:      1-5) 


Summary 
In  summary,    it  can  be  seen   from  a  review  of   the 
descriptive   information  presented   in   this   chapter    that   the 
study  sample  represented  considerable  diversity  among  organiza- 
tions  and  projects.      The  organizations    in   the  sample  were   from 
varied   industries;    they  represented  a  wide  range   in   terms   of 
size;    and   they  differed   in  varying  degrees   on  several   other 
organizational  attributes   relevant   to   the  study.      With   respect 
to    the   individual   projects   studied,    there  were    three  general 
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types  of  MIS  projects:   models;  projects  that  were  spinoffs 
from  data  processing  applications;  and  projects  which  were 
developed  from  the  ground  up  to  provide  data  collection  and 
analysis  capability  in  specified  planning  and  control  areas. 
The  twenty  projects  represented  substantial  diversity  in 
terms  of  size  and  scope;  procedural  and  organizational 
attributes;  and,  finally,  performance  on  the  criteria  of 
success . 
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CHAPTER  VI 
TESTS   OF   HYPOTHESES 

Introduction 

This   chapter  and   the  one  which   follows   are  devoted 
to  presenting   the  results   of   the  analysis   of   the  sample  data. 
As    indicated   in  Chapter   IV,    the  primary   form  of  statistical 
analysis  was    the  computation  of  Kendall's  rank  correlation 
statistic,    tau.      This  measure  of   the  degree  of  relationship 
between   the  rankings   of   two  variables   was    tested   for  statistical 
significance   in  all   cases. 

The  number  of  possible  cross-classifications,    and 
resultant  relationship  statistics,  with  61   variables  was   3660. 
To  avoid   burdening   the  reader  with  all  of   these  statistics, 
and   to  conserve  computation   time  and  cost,    the  steps   discussed 
in  Chapter   IV  were   followed.      The  results   of   the  data  analysis 
have  been  further   limited  here  by  presenting  only   the   following: 

1.  Relationships   among  criterion  variables; 
all    tau  statistics   presented  regardless 
of  significance   levels. 

2.  Relationships   of  hypothesis   variables 

to  each  of   the   four  criteria  of   success; 
all    tau  statistics   presented  regardless 
of  significance   levels. 

3.  Relationships   of  criterion  variables    to 
other  non-hypothesis   variables   which  were 
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signif leant  at  the  .10  level  or  beyond. 

4.  Relationships  of  hypothesis  variables 
to  other  non-criterion  variables  which 
were  significant  at  the  .10  level  or 
beyond. 

5.  Other  relationships  among  variables 
felt  by  the  researcher  to  be  relevant 
to  the  study,  which  contributed  to 
understanding  of  the  overall  conclusions 
drawn,  and  which  were  significant  at  the 
.10  level  or  beyond.   This  last  category 
is  presented  in  Chapter  VII. 

All  tests  for  the  significance  of  the  relationship 
between  the  rankings  of  two  variables  were  made  with  the  .10 
level  as  the  cutoff.   Any  relationship  with  a  probability  of 
chance  occurrence  greater  than  .10  was  not  considered  to  be 
significant.   Since  the  direction  of  expected  relationship 
was  stated  in  only  those  cases  where  a  hypothesis  was  tested 
(category  2  above),  all  significance  tests  were  two-tailed 
tests  except  those  involving  the  relationship  of  a  hypothesis 
variable  to  a  criterion  variable. 

Relationships  Among  Criterion  Variables 
At  the  time  this  study  was  conceived,  it  was  assumed 
that  the  four  criteria  to  be  used  were  independent.   Otherwise, 
all  four  would  not  have  been  included  in  the  study.   However, 
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no  hypothesis  to  this  effect  was  set  up.  For  this  reason, 
all  tests  of  relationship  among  the  four  criteria  are  two- 
tailed   tests. 

It  can  be  seen  from  Table   15   that   there  were  no 
relationships   among  any  of   the  criteria  which   reached  signifi- 
cance at   the    .10   level.      This  supports    the  assumption  of 
independence  among   the   four  criteria  used    in   the  study. 

TABLE    15 
Relationships   among  Criterion  Variables 


Cost 

(Var. 

51) 

User  Satis- 
faction  (Var.    54) 

Oper* 
Prob] 

itional 
.ems    (V. 

ar.    59) 

Tau 

Z(s) 

P(Z)2 

Tau 

Z(s) 

P(Z)2 

Tau 

Z(s) 

P(Z)2 

Time 
(Var.   48) 

.257 

1.534 

1 

.124 

.064 

.359 

.720 

.067 

.281 

.778 

Cost 
(Var.    51) 

.091 

.522 

.604 

-.030 

.105 

.916 

User   Sat- 
isfaction 
(Var.    54) 

.345 

1.582 

.104 

All  values   shown   for   tau  in  all   tables   are   tau-C  unless   a  given 
value   is   prefixed  by  B,    in  which  case   that  value   is    tau-B. 
See  Appendix  F   for  differences    in  computation  of   tau-B  and 
tau-C. 

'P(Z)   value  shown   includes   both    tails   of    the  normal  distribution, 
For  all   tables,    the  value  shown  under   P(Z)   represents    the 
chance  probability,   one-tailed  or   two-tailed,   as    indicated, 
of  having  a  normal   deviate   for   the  S   statistic   as    large   as 
the  one  shown  under   Z(s). 
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The  strongest  relationship  between  any  two  criterion 
variables  was  the  one  between  user  satisfaction-user  (Var.  54) 
and  operations  problems  (Var.  59),  with  a  tau  of  .345.   This 
relationship  had  a  chance  probability  of  .104,  which  almost 
reached  the  .10  cutoff  level  established.   Although  not  signifi- 
cant based  on  the  arbitrary  .10  cutoff  used  in  the  study,  there 
did  appear  to  be  a  meaningful  relationship  between  how  satisfied 
the  user  was  and  how  successfully  the  project  had  been  implemented 
by  computer  operations.   Where  there  were  substantial  problems 
for  computer  operations  in  running  particular  programs,  or  in 
meeting  certain  schedules,  there  also  tended  to  be  lower  user 
satisfaction.   It  cannot  be  assumed  that  computer  operations 
difficulties  caused  user  dissatisfaction,  although  this  did 
appear  to  be  the  case  with  two  of  the  projects  studied.   Rather, 
the  conclusion  drawn  was  that  those  projects  which  created 
problems  for  the  computer  operations  area  were  also  somewhat 
unsatisfactory  in  the  eyes  of  the  users  of  the  outputs.   In 
other  words,  there  was  a  tendency  for  shortcomings  in  the 
systems  development  work  for  these  projects  to  be  reflected 
in  both  inadequate  planning  for  impact  on  computer  operations 
and  poor  response  to  user  information  needs. 

Tests  of  Hypotheses 
Tables  16-19  provide  a  complete  picture  of  the  relation- 
ship between  each  hypothesis  variable  and  each  criterion  variable. 


TABLE  16 

Success  in  Terms  of  Time 

Actual  Time/Estimated  Time  (Var.  48) 
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Rank  Var. 

1  User  Participation-User  46 

2  Measurable  Project  Objectives  12 

3  Project  Team  14 

7  Level  of  Information  Systems 
Manager  9 

8  Two  or  More  Years  Systems 
Experience  27 

12  Documentation  Standards  35 

13  Project  Control  42 

14  Turnover  29 

16  Originator  11 

17  Centralization  8 

20  Two  or  More  Years  Organization 

Experience  34 

23  High  Level  Programming  Language  43 

31  Mean  Years  Formal  Education  32 

32  Combination  Analyst/ Programmer  23 

33  Systems  Staff/Total  Employees  61 

34  Hardware  Investment/Sales  6 


Tau 


21*1. 


mi 


-.181  1.061  .145 

-.219  1.142  .127 

-.130  .477  .316 


-.292*        1.358 


.087 


.203  1.193  .116 

-.380*        1.753  .040 

-.153  .763  .223 

.082  .529  .300 

-.037  .155  .439 

Not   tested  by   this  sample 


.297* 

1.740 

.041 

-.315* 

1.774 

.039 

.630* 

3.471 

.0003 

.420* 

1.562 

.060 

.291* 

1.659 

.050 

-.063 

.330 

.378 

*   -  Significant  at   the   .10   level  or  beyond   -  one-tailed. 


TABLE  17 
Success  in  Terms  of  Cost 
Actual  Cost/Estimated  Cost  (Var.  51) 
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Rank 

1  User  Participation -User 

2  Measurable  Project  Objectives 

3  Project  Team 

7     Level  of   Information  Systems 
Manager 


Var. 


Tau 


Zisl 


an. 


46 

.154 

.894 

.185 

12 

.100 

.503 

.309 

14 

.210 

.794 

.214 

-.240 


1.107 


.135 


8 

Two  or  More  Years  Systems 

Experience 

27 

-.104 

.596 

.275 

12 

Documentation  Standards 

35 

-.200 

.899 

.185 

13 

Project  Control 

42 

-.220 

1.109 

.134 

14 

Turnover 

29 

.030 

.158 

.437 

16 

Originator 

11 

.187 

.930 

.177 

17 

Centralization 

8 

Not  tested 

by  this 

sample 

20 

Two  or  More  Years  Organization 

Experience 

34 

.055 

.295 

.384 

23 

High  Level  Programming  Language 

43 

-.097 

.519 

.302 

31 

Mean  Years  Formal  Education 

32 

.108 

.567 

.287 

32 

Combination  Analyst/ Programmer 

23 

-.010 

.000 

.500 

33 

Systems  Staff/Total  Employees 

61 

.263* 

1.491 

.069 

34 

Hardware  Investment/Sales 
Significant  at  the  .10  level  or 

6 
bey 

-.023 

.099 

.461 

*  . 

ond  -  one-tailed. 
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TABLE   18 
Success    in  Terms  of  User   Satisfaction 
User  Satisfaction-User   (Var.    54) 


Rank 

1  User  Participation-User 

2  Measurable  Project  Objectives 

3  Project  Team 

7  Level  of   Information  Systems 
Manager 

8  Two  or  More  Years   Systems 


Var. 


Tau 


zisi 


.105 


.464 


lizi 


46 

.440* 

2.618 

.005 

12 

-.081 

.403 

.345 

14 

.080 

.278 

.390 

.322 


Experience 

27 

-.126 

.729 

.234 

12 

Documentation  Standards 

35 

.090 

.379 

.352 

13 

Project  Control 

42 

-.207 

1.040 

.149 

14 

Turnover 

29 

-.217* 

1.480 

.069 

16 

Originator 

11 

.420* 

2.132 

.017 

17 

Centralization 

8 

Not  tested 

by  this 

sample 

20 

Two  or  More  Years  Organization 

Experience 

34 

.253* 

1.477 

.069 

23 

High  Level  Programming  Language 

43 

.067 

.346 

.366 

31 

Mean  Years  Formal  Education 

32 

.222 

1.201 

.115 

32 

Combination  Analyst/ Programmer 

23 

.420* 

1.561 

.060 

33 

Systems  Staff /Total  Employees 

61 

.229* 

1.293 

.098 

34 

Hardware  Investment/ Sales 

6 

.057 

.296 

.383 

*  -  Significant  at   the    .10   level  or  beyond   -  one-tailed. 


TABLE  19 

Success  in  Terms  of  Computer  Operations 
Computer  Operations  Problems  (Var.  59) 
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Rank 


Var. 


Tau 


Z(s) 


mi 


1 

User  Participation-User 

46 

.180 

.820 

.206 

2 

Measurable  Project  Objectives 

12 

-.007 

.000 

.500 

3 

Project  Team 

14 

-.050 

.170 

.433 

7 

Level  of  Information  Systems 

Manager 

9 

B-.285* 

1.340 

.090 

8 

Two  or  More  Years  Systems 

Experience 

27 

-.255 

1.176 

.120 

12 

Documentation  Standards 

35 

-.170 

.811 

.210 

13 

Project  Control 

42 

-.060 

.261 

.397 

14 

Turnover 

29 

B  .000 

.000 

.500 

16 

Originator 

11 

B-.147 

.665 

.254 

17 

Centralization 

8 

Not  tested 

by  this 

sample 

20 

Two  or  More  Years  Organization 

Experience 

34 

.157 

.707 

.241 

23 

High  Level  Programming  Language 

43 

B-.157 

.695 

.243 

31 

Mean  Years  Formal  Education 

32 

.330* 

1.542 

.062 

32 

Combination  Analyst/Programmer 

23 

.060 

.204 

.420 

33 

Systems  Staff /Total  Employees 

61 

.037 

.143 

.443 

34 

Hardware  In vestment /Sales 
Significant  at  the  .10  level  or 

6 
bey 

.232 

1.063 

.144 

*  . 

ond  -  one- 

tailed. 
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Each  of  these  tables  shows  the  relationships  between  all  sixteen 
of  the  variables  representing  hypotheses  to  be  tested  and  one 
criterion  variable.   The  sixteen  hypothesis  variables  are  shown 
in  the  same  order  as  in  the  ranking  in  Chapter  IV;  the  number 
to  the  left  of  each  variable  is  the  rank  of  that  variable  from 
the  original  list  (Figure  2,  Chapter  IV). 

Rather  than  merely  repeat  in  narrative  form  what  is 
contained  in  Tables  16-19,  the  comments  here  will  be  restricted 
to  general  observations  about  the  findings  shown  in  the  tables, 
and  discussions  of  those  hypotheses  which  were  significantly 
related  to  at  least  one  criterion: 

1,   Each  of  the  hypotheses  was  represented  by  one 
variable  in  the  statistical  analysis.   Appendix 
E  should  be  consulted  for  a  description  of  what 
each  variable  is  a  measure  of,  and  how  it  is 
scored.   Of  the  sixteen  hypotheses,  all  but 
one  were  tested  by  the  data  from  the  sample. 
Variable  8,  representing  the  degree  of  centrali- 
zation of  the  information  systems  function,  was 
dropped  from  the  analysis  because  of  the  lack  of 
spread  on  the  variable  among  the  organizations  in 
the  sample.   Eight  organizations  were  scored  at 
the  highest  level  of  centralization,  and  the 
remaining  two  were  at  the  next  to  highest  level, 
a  result  solely  of  small  operations  research 
groups  which  did  some  programming  work. 
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2.  Of  the  fifteen  hypotheses  which  were  tested, 
five  were  found  not  significantly  related  to 
any  criterion.   Those  five  were: 

2  -  Measurable  project  objectives  -  Var.  12 

3  -  Project  team  Var.  14 

8  -  Two  or  more  years  systems 

experience  Var.  27 

13  -  Project  control  Var.  42 

34  -  Hardware  investment/ sales       Var.  6 

The  fact  that  those  hypotheses  which  were  not 

significantly  related  to  any  criterion  were  from 

the  top,  middle,  and  bottom  of  the  ranked  listing 

is  of  of  some  interest.   At  least  for  the  sample 

involved  in  this  study,  those  factors  believed 

by  the  SMIS  conference  respondents  to  be  most 

crucial  to  MIS  project  success  did  not  prove  to 

be  so  critical  in  the  case  of  two  of  the  three 

top  ranked  factors.   Also  of  considerable  interest 

was  the  lack  of  significant  relationship  between 

reported  project  control  efforts  and  any  criterion 

of  success  used  in  the  study.   This  last  finding 

will  be  explored  further  at  a  later  point. 

3.  Of  the  ten  hypothesized  relationships  which  were 
found  to  be  significant  at  the  .10  level  or 
beyond,  four  were  cases  where  a  hypothesis  variable 
was  significantly  related  to  only  one  criterion 


-100- 

variable.   The  remaining  six  hypothesis 
variables  were  related  to  two  or  three 
criterion  variables. 
4.   Looking  at  the  tables  from  a  criterion 
perspective,  the  time  success  criterion 
was  significantly  related  to  more  hypothesis 
variables  (7)  than  was  any  other  criterion 
variable.   User  satisfaction  was  signifi- 
cantly related  to  six  hypothesis  variables, 
computer  operations  success  to  two,  while 
cost  success  was  significantly  related  to 
only  one  hypothesis  variable. 

Hypotheses  Significantly  Related  to  Project  Success 

Each  of  the  hypothesis  variables  which  was  significantly 

related  to  at  least  one  criterion  of  project  success  is  dealt 

with  briefly  below: 

1  -  User  Participation-User  (Var.  46).   As  hypothesized, 
the  higher  the  level  of  perceived  user  participation 
in  design  and  development  of  a  project,  the  greater 
the  success  of  that  project  as  viewed  by  the  user. 
The  statistical  relationship  between  these  two 
variables  was  very  strong  In  the  sample.   However, 
perceived  user  participation  in  project  develop- 
ment was  not  significantly  related  to  any  of  the 
other  three  criteria  of  success. 
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7  -  Level  of  Information  Systems  Manager  (Var.  9). 
The  higher  the  level  of  managers  of  the  infor- 
mation systems  departments  in  the  organizations 
sampled,  the  less  successful  projects  were  in 
terms  of  time,  and  the  more  successful  they  were 
in  terms  of  computer  operations.   Although  there 
were  no  data  from  the  study  which  explained  these 
relationships  very  satisfactorily,  it  could  be 
that  the  higher  the  level  of  the  top  computer 
executive  in  the  organization,  the  less  the 
felt  pressure  in  the  information  systems  function 
to  meet  time  budgets.   With  respect  to  computer 
operations  problems  being  less  where  the  top 
computer  executive  was  at  a  higher  level  in  the 
organization,  one  possible  explanation  would  be 
the  apparent  greater  attention  to  computer 
operations  requirements  when  the  information 
systems  function  is  accorded  higher  status  in 
the  organization.   Put  differently,  where  the 
organization  has  placed  emphasis  on  using  the 
computer  effectively,  as  evidenced  by  locating 
the  top  computer  executive  close  to  top  manage- 
ment, there  may  be  more  emphasis  on  assuring 
that  computer  operations  requirements  are 
considered  in  systems  design. 
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12  -  Documentation  Standards  (Var.  35).   Where 

formal  documentation  standards  were  reported 
to  have  been  applicable  to  projects  in  the 
study,  the  time  success  tended  to  be  greater 
than  where  there  were  no  such  standards.   It 
was  impossible  to  determine  whether  the  documen- 
tation standards  themselves  actually  contributed 
to  better  performance  in  terms  of  time,  or  the 
existence  of  such  standards  merely  reflected 
an  atmosphere  of  greater  control  over  informa- 
tion systems  activities.   In  any  case,  this 
finding  should  be  interpreted  with  caution  due 
to  the  great  extent  of  ties  in  the  ranking  of 
the  dlchotomous  documentation  standards  variable. 

14  -  Turnover  (Var.  29).   As  was  pointed  out  in 

Chapter  V,  only  three  of  the  projects  in  the 
sample  experienced  any  turnover  of  project 
personnel.   Therefore,  the  negative  relationship 
of  turnover  to  user  satisfaction  must  be  Inter- 
preted very  cautiously  due  to  the  great  extent  of 
ties  in  the  turnover  ranking.   A  close  review 
of  all  the  variables  for  those  three  projects 
with  turnover  led  to  the  conclusion  that  the 
negative  relationship  with  user  satisfaction 
was  a  chance  one,  and  should  not  be  given  much 
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credence.   The  reason  for  this  conclusion 
was  that  In  the  case  of  only  one  project 
did  low  user  satisfaction  follow  at  all 
from  project  staff  turnover.   In  that  one 
case,  the  turnover  was  among  key  user  repre- 
sentatives on  the  project  team. 

16  -  Originator  (Var.  11).   The  hypothesis  that 
success  is  greater  for  projects  originated 
by  users,  as  opposed  to  those  initiated  by 
top  management  or  the  Information  systems 
staff,  was  confirmed  where  success  was  viewed 
in  terms  of  user  satisfaction.  This  would 
appear  to  stem  in  part  from  a  greater  feeling 
of  participation  among  the  users  who  originated 
projects,  and,  in  part,  from  a  greater  clarity 
in  what  the  users  wanted  when  they  Initiated 
projects  themselves.  Where  the  users  knew 
what  they  wanted,  and  participated  In  develop- 
ing it,  their  satisfaction  with  the  results 
tended  to  be  greater. 

20  -  Two  or  More  Years  Organ 1 gat ion  Experience  (Var.  34). 
The  organization  experience  of  the  project  staff 
was  significantly  related  to  two  criteria:   time 
and  user  satisfaction.   In  the  case  of  time,  the 
greater  the  organisation  experience  of  the  project 
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staff,  the  poorer  the  time  performance. 
This  would  seem  to  reflect  a  greater  tendency 
for  those  who  knew  their  organizations  well 
to  spend  more  time  delving  into  the  various 
ramifications  and  possibilities  with  projects 
once  development  was  underway.   This  would 
likely  draw  out  the  time  spent  on  the  projects 
beyond  what  was  originally  anticipated.   In 
the  case  of  user  satisfaction,  the  positive 
relationship  to  organization  experience  of 
the  project  staff  indicated  that  the  greater 
the  proportion  of  those  working  on  a  project 
who  understood  the  organization  they  were  serving, 
the  better  the  results  in  the  eyes  of  the  users. 
It  is  worth  noting  here  that  organization 
experience  was  not  related  at  all  to  the 
measure  of  user  participation.   It  might 
have  been  assumed  that  high  organization 
experience  would  have  reflected  high  propor- 
tions of  user  representatives  on  project  teams, 
which,  in  turn,  would  have  contributed  to 
high  levels  of  perceived  user  participation. 
Since  user  participation  was  highly  related 
to  user  satisfaction,  the  assumption  would 
then  be  that  this  last  relationship  was 
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really  what  was  being  picked  up  in  a 
relationship  between  user  satisfaction 
and  organization  experience  of  the 
project  staff.   This  was  not  the  case, 
however. 

23  -  High  Level  Programming  Language  (Var.  43). 
From  the  data  in  the  sample,  the  hypothesis 
of  greater  project  success  with  higher  level 
programming  languages  was  confirmed  with 
respect  to  the  time  criterion.   Where  COBOL 
was  the  programming  language  used,  the 
projects  were  completed  in  less  time 
relative  to  estimates  than  where  FORTRAN  or 
assembly  language  was  used.   However,  it 
should  be  noted  that  the  types  of  projects 
were  different  where  FORTRAN  was  used,  and 
this  in  itself  influenced  the  time  performance. 
FORTRAN  was  used  with  modeling  applications 
which  were  evolutionary  in  nature.   Through 
trial  and  error  processes,  the  models  were 
developed  to  the  point  the  users  were  satis- 
fied with  the  results.   This  situation 
resulted  in  very  extended  development  periods 
for  these  projects. 

31  -  Mean  Years  Formal  Education  (Var.  32).   The 

mean  education  level  of  the  project  staff  was 
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very  strongly  related  to  the  time  criterion 
of  success.  What  this  indicated  was  that 
the  higher  the  education  of  those  who 
worked  on  projects,  the  poorer  the  perform- 
ance in  terms  of  time.   The  only  apparent 
explanation  for  this  relationship  was  a 
tendency  for  those  with  more  formal 
education  to  delve  more  deeply  into 
possible  enhancements  and  embellishments 
on  the  projects  they  worked  on.   Again,  the 
influence  of  modeling  applications  should 
be  considered  in  this  connection.   Those 
engaged  in  modeling  applications  generally 
had  high  levels  of  formal  education.   It 
will  be  recalled  that  the  modeling  projects, 
because  of  their  evolutionary  nature,  were 
drawn  out  and  usually  way  over  the  original 
estimates  for  time  to  completion. 
Also  of  significance  was  the  relationship  of 
project  staff  education  levels  to  computer 
operations  success.   One  explanation  for 
this  relationship  appeared  to  be  the  tendency 
for  those  with  the  greatest  amounts  of  formal 
education  to  make  provisions  for  the  efficient 
computer  implementation  of  their  projects. 
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32  -  Combination  Analyst/Programmer  (Var.  23). 

Where  combination  analyst/programmers  were 
used  to  develop  projects,  the  time  performance 
was  relatively  poor,  but  user  satisfaction  was 
relatively  high.  While  no  explanation  was 
readily  apparent  for  the  poor  time  performance 
where  combination  analyst/programmers  were 
used,  the  high  user  satisfaction  seemed  to 
be  attributable  to  the  ability  of  the  user 
to  look  to  one  person  for  any  problem  that 
arose.  Where  the  user  had  to  deal  with  an 
analyst  in  some  cases,  and  with  a  programmer 
in  others,  there  appeared  to  be  user  frustra- 
tion and  less  satisfaction  with  the  project 
as  a  whole. 

33  -  Systems  Staff /Total  Employees  (Var.  61). 

Although  they  were  all  rather  weak  relationships, 
the  relative  size  of  the  organization's  systems 
staff  was  related  to  three  separate  criteria. 
Positive  relationships  to  both  the  time  and 
the  cost  criteria  would  seem  to  indicate  that 
there  was  more  slack  in  those  organizations 
with  large  systems  staffs.  Where  large  staffs 
existed,  the  pressure  to  meet  time  and  cost 
budgets  or  estimates  did  not  seem  to  be  as  great. 
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As  a  consequence,  time  and  cost  performance 
were  poor.   However,  user  satisfaction  was 
higher  for  those  projects  developed  in  organi- 
zations with  larger  systems  staffs.   Taken 
all  together,  the  pattern  would  seem  to  be  one 
of  substantial  organizational  commitments  to 
effective  use  of  the  computer  as  an  aid  to 
managerial  dec is  ion -making,  manifested  through 
relatively  large  systems  staffs  which  were 
oriented  toward  giving  the  users  what  they 
wanted  at  the  expense  of  time  and  cost  overruns. 

Further  Comments  Concerning  the  Criteria  of  Success 
Data  were  collected  for  several  other  variables 
besides  the  criteria  and  the  hypotheses  to  be  tested.   To  provide 
a  more  complete  picture,  all  significant  relationships  between 
these  other  variables  and  the  criterion  variables  are  shown  in 
Tables  20-23.   These  tables,  and  certain  relevant  non-quantitative 
findings,  are  discussed  in  this  section. 

Time  Success  (Var.  48).  Table  20  contains  those 
relationships  between  time  success  and  other  variables  in  the 
study  which  were  significant  at  or  beyond  the  .10  level.   As 
might  be  expected,  the  projects  with  the  greatest  elapsed  time 
had  the  poorest  time  success.   Also,  the  higher  the  proportion 
of  college  degrees  on  project  staffs,  the  poorer  the  time  perform- 
ance.  This  is  merely  another  way  of  looking  at  the  educational 
level,  which  was  discussed  earlier. 
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TABLE   20 
Actual   Time/Estimated  Time    (Var.   48) 
(Two-tailed  significance  at  or  beyond   .10  only) 

Var.  Tau  Z(s)  P(Z) 


College  Degree 

30 

.550 

3.281 

.0014 

Elapsed  Months 

21 

.515 

3.081 

.002 

User  Management  Interest 

39 

-.367 

1.864 

.062 

Time  Success -PL 

49 

-.437 

2.328 

.020 

From  the  data  in  the  study,  it  appeared  that  high 
interest  and  involvement  of  top  level  user  management  contributed 
to  better  time  performance.   Where  the  top  management  levels  of 
user  organizations  were  actively  engaged  in  promoting  projects, 
those  projects  seemed  to  get  completed  faster  relative  to  time 
estimates.   This  finding  has  interesting  implications  which 
tie  in  to  some  comments  to  be  made  later  concerning  the  reward 
structures  for  systems  people  in  organizations. 

Another  finding  shown  in  Table  20  was  the  accuracy 
of  project  leader's  perceptions  concerning  the  success  of  their 
projects  in  terms  of  time.   It  should  be  borne  in  mind,  however, 
that  this  relationship  is  based  on  asking  the  project  leader 
only  how  successful  he  felt  the  project  was  in  terms  of  time, 
with  no  consideration  given  to  his  views  of  other  aspects  of 
project  success.   This  point,  too,  will  be  dealt  with  later 
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at  greater  length. 

Cost  Success  (Var.  51).   Table  21,  consisting  of 
only  one  entry,  provides  the  same  sort  of  confirmation  of 
the  project  leaders'  perceptions  about  project  cost  success 
as  described  just  above  for  time  success.   Again,  this  point 
will  be  dealt  with  later  in  the  context  of  global  project 
success,  and  what  that  seemed  to  be  comprised  of  in  the 
eyes  of  project  leaders. 

TABLE  21 
Actual  Cost/Estimated  Cost  (Var.  51) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 
Cost  Success -PL  52    -.644    3.444    .0006 


In  addition  to  the  questions  asked  of  project  leaders 
which  were  tied  to  some  kind  of  response  scale,  there  were  two- 
open-ended  questions  which  permitted  project  leaders  to  state 
in  their  own  words  the  reasons  they  felt  their  projects  came 
out  poorly  on  time  and/or  cost  performance.   These  comments 
were  reviewed  by  the  researcher  and  classified  according  to 
eight  response  categories.   The  categories,  and  the  number  of 
projects  falling  into  each,  were  as  follows:   (there  were 
multiple  responses  for  most  projects) 
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1.  Poor  initial  estimates;  especially 
inadequate  was  the  time  allowed  for 
implementation 10 

2.  Inexperience  of  the  project  staff 
with  this  particular  type  of  applica- 
tion or  language 7 

3.  Key  people  on  the  project  were  doing 
several  things  at  once;  competing 

demands  for  their  time  caused  delays  4 

4.  The  project  was  allowed  to  evolve  over 
a  long  period  of  user  learning  and 
growth;  no  attempt  was  made  to  freeze 
requirements  at  one  point  and  then 
consider  the  job  done  when  those 
requirements  were  programmed  4 

5.  File  handling  problems;  delays  caused 
by  waiting  for  data  base  development 

or  accessibility  3 

6.  Poor  computer  test  turn-around  time 2 

7.  Turnover  of  project  staff 0  .  2 

8.  Project  was  too  large  and  complex 

to  be  managed. .2 

From  the  above,  it  would  appear  that  the  greatest 
problem  in  meeting  time  and  cost  budgets,  at  least,  in  the 
eyes  of  project  leaders,  was  poor  estimates  of  time  and  cost 
in  the  first  place.   Actually,  most  of  the  other  categories 
are  related  in  some  way  to  the  first  one.   In  estimating  time 
to  complete  a  project,  the  experience  of  the  project  staff 
should  have  been  considered,  as  should  the  expected  availability 
of  computer  test  time,  and  so  forth.   This  is  not  to  say  that 
every  contingency  could  have  been  foreseen  and  provided  for 
in  making  initial  estimates,  but  it  would  seem  that  some  factors 
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which  did  cause  delays  were  not  given  adequate  consideration 
at  the  time  estimates  were  made. 

One  final  point  concerning  project  leaders1  comments 
about  time  and  cost  performance  is  worth  mention.   In  two 
cases,  the  project  leader  had  no  direct  authority  over  programmers. 
This  created  communications  problems  which  resulted  in  considerable 
confusion  as  to  what  was  to  be  programmed,  with  attendant  delays. 
Both  of  the  projects  were  in  the  lower  half  of  every  criterion 
ranking. 

User  Satisfaction  (Var.  54).   A  close  scrutiny  of  the 
users'  ratings  of  their  satisfaction  with  the  results  of  the 
projects  in  the  study  revealed  several  interesting  points.   It 
will  be  recalled  that  the  measure  of  user  satisfaction  was  a 
composite  variable,  made  up  of  five  separate  items  which  were 
highly  intercorrelated.   However,  one  of  the  components  of 
the  user  satisfaction  variable,  a  measure  of  implementation 
problems,  was  also  treated  as  a  separate  variable  (53)  for 
comparative  purposes.   As  Table  22  shows,  the  users  viewed 
the  ease  or  difficulty  of  implementing  a  completed  project 
as  a  very  important  aspect  of  the  perception  of  project  success. 


While  there  is  bias  in  the  value  of  tau  shown  in  Table 
22,  due  to  the  inclusion  of  the  implementation  problems 
variable  in  the  user  satisfaction  variable,  it  is  apparent 
that  implementation  was  a  large  factor  in  the  minds  of  users 
in  evaluating  the  success  of  projects. 
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TABLE  22 
User  Satisfaction-User  (Var.  54) 
(Two-tailed  significance  at  or  beyond  .10  only) 


Var, 


Tau     Z(s)     P(Z) 


Implementation  Problems- 
User 

User  Satisfaction-PL 

Project  Success-PL 

Computer  Operations 
Cost 

Satisfaction  of  Objectives- 
PL 

Man  Months 


53 

.720 

4.192 

.0004 

57 

.656 

3.584 

.0004 

56 

.607 

3.124 

.002 

60 

55 
22 


.347  1.796 


.074 


.320  1.687  .092 

.280  1.668  .096 


It   is   also  noteworthy  that   the  variable  representing 
the  project   leader's   perception  of   implementation  problems    (58) 
was  not  significantly  related  to  user  satisfaction.      This   last 
observation  will  be  pursued   further  at  a   later  point,   but   it 
is   apparent   that  users   viewed   implementation  problems  differently 
than  project   leaders   viewed   them.      This   conclusion   is   supported 
by   the  very  strong  relationships   between  user  satisfaction  and 
three  variables   representing   the  project   leader's   evaluation 
of  project  success.      These   three  variables,   shown   in  Table   22, 
were   the  project   leader's   perception  of  how  satisfied    the 
users   were  with    the   project  results    (57),   how  successful    the 
prajwct   leader   felt   the  project  was  overall   (56),   and  how  well 
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the  project  leader  thought  the  original  objectives  for  the 
project  had  been  satisfied  (55).   In  other  words,  users  and 
project  leaders  agreed  very  well  on  the  relative  success  of 
a  project,  but  implementation  problems  were  a  factor  in  the 
users'  evaluations  while  they  were  not  a  factor  in  the  project 
leader's  evaluation  of  project  success. 

Another  very  interesting  aspect  of  user  satisfaction 
came  to  light  in  looking  at  the  relationships  among  the 
various  criteria.   It  was  felt  that  users  might  have  been 
dissatisfied  with  projects  which  took  too  long  to  complete; 
that  they  disliked  waiting  for  something  they  felt  they  needed. 
To  get  at  this  question,  the  individual  projects  that  had 
the  worst  performance  on  time  were  checked  for  user  satisfaction 
scores.   It  was  already  known  that  these  two  criteria  were 
not  significantly  related  for  the  whole  study,  but  it  was  felt 
that  looking  at  the  extremes  might  reveal  something  that  was 
hidden  in  the  sample  statistics.   The  surprising  result  of 
this  investigation  was  that  the  project  ranked  last  on  time 
success  was  ranked  third  on  user  satisfaction,  and  the  project 
ranked  next  to  last  on  time  success  was  ranked  second  on  user 
satisfaction.   In  other  words,  two  of  the  top  three  projects 
in  terms  of  user  satisfaction  were  the  two  bottom  projects 
on  time  success.   Upon  closer  review,  what  appeared  to  be 
underlying  this  finding  was  the  evolutionary  nature  of  the 
two  projects  in  question.   Both  projects  involved  rather 
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complex  models  which  were  developed  over  a  fairly  long  period 
of  trial  and  error.   Rather  than  taking  a  set  of  user  stated 
desires  or  needs,  and  developing  some  programs  to  satisfy  them, 
the  systems  staffs  concerned  worked  very  closely  with  the  users 
in  evolving  those  products  which  best  fit  the  users'  needs. 
There  seemed  to  have  been  tacit  recognition  that  the  users 
would  learn  as  they  went,  that  they  would  become  more  sophis- 
ticated as  they  worked  with  outputs  at  successive  stages  of 
development,  and  would,  therefore,  not  be  satisfied  with  a  one- 
shot  development  effort.   Although  the  users  were  highly 
satisfied  with  what  they  ultimately  received  from  the  projects, 
this  evolutionary  approach  took  much  longer  than  had  originally 
been  anticipated. 

Based  on  the  above  findings,  an  effort  was  made  to 
discover  other  evidence  in  the  data  which  supported  what  has 
been  called  the  evolutionary  approach  to  project  development. 
It  was  hypothesized  that  projects  which  had  not  been  developed 
with  such  an  evolutionary  strategy  would  reflect  the  following 
characteristics : 

Users  who  were  originally  satisfied  with  project 
results  at  the  time  the  projects  were  completed 
might  now  be  less  satisfied  with  what  they  were 
receiving.   This  shift  in  satisfaction  over  time 
would  have  resulted  from  a  user  learning  process 
which  was  not  matched  by  any  enhancements  in  the 
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inforraation  system  outputs  they  were  receiving. 
In  other  words,  as  the  users  became  more  sophis- 
ticated through  working  with  the  products  of  the 
projects  in  question,  their  information  needs 
would  shift.   If  the  information  system  products 
did  not  shift  with  this  increasing  sophistication, 
the  users  would  now  be  less  satisfied  with  the 
projects  than  when  they  were  first  implemented. 

To  test  the  above  hypothesis,  two  separate  components 
of  the  user  satisfaction  variable  were  analyzed.   These 
components  represented  a  measure  of  how  well  the  user  felt  the 
original  project  objectives  had  been  met,  and  a  measure  of  how 
satisfied  the  user  now  was  with  the  products  of  the  project. 
It  was  already  known  that  these  two  components  of  overall  user 
satisfaction  were  highly  correlated.   However,  it  was  not  known 
if  these  two  factors  were  scored  at  about  the  same  level  on  a 
project  by  project  basis.   To  get  at  this  latter  question,  the 
mean  value  and  the  standard  deviation  were  computed  for  each 
factor.   These  statistics  were  then  used  in  testing  the  differ- 
ence between  the  two  means  to  determine  its  significance.   The 
mean  value  for  the  factor  measuring  how  well  original  objectives 
were  reportedly  met  was  39.2  with  a  standard  deviation  of  9.8. 
The  mean  and  standard  deviation  for  the  reported  current  satis- 
faction factor  were  33.5  and  10.3,  respectively.   The  difference 
between  these  two  means  was  5.7,  which,  when  tested  by  the  t 
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statistic  with  38  degrees  of  freedom,  was  significant  beyond 
the  .10  level  for  a  two-tailed  test. 

Although  the  t  test  described  above  was  retrospective, 
and,  therefore,  not  general tzable  beyond  the  sample,  it  was 
interpreted  as  an  indication  that  the  users  in  the  study  did  shift 
their  perceived  information  requirements  fairly  quickly  after  MIS 
project  implementation.   While  users  may  have  scored  a  project 
very  high  on  how  well  it  served  their  information  needs  when 
first  implemented,  they  tended  to  score  present  satisfaction  at 
a  lower  level.   This  interpretation  would  seem  to  be  a  good 
argument  for  the  evolutionary  approach  to  MIS  projects  since 
such  projects  have  been  defined  as  management -dec is ion  oriented. 

Finally,  with  respect  to  user  satisfaction,  Appendix  G 
contains  comments  by  users  relative  to  what  they  believed  should 
have  been  done  to  improve  the  projects  in  the  study,  and  what 
they  view  present  shortcomings  to  be.   These  comments  are  not 
direct  quotes;  rather,  they  represent  the  notes  taken  by  the 
researcher  as  the  users  responded  to  an  open-ended  question  in 
the  questionnaire  (Appendix  B,  page  222). 

Computer  Operations  Problems  (Var.  59).   As  reflected 
in  Table  23,  the  measure  of  computer  operations  success  used  in 
the  study  was  related  to  a  secondary  measure  of  such  success, 
namely,  the  operations  cost  of  a  project.   This  relationship 
indicates  that  projects  which  were  Implemented  most  smoothly 
by  computer  operations  were  also  the  most  successful  in  terms 


-117- 

statistic  with  38  degrees  of  freedom,  was  significant  beyond 
the  .10  level  for  a  two-tailed  test. 

Although  the  t  test  described  above  was  retrospective, 
and,  therefore,  not  general izable  beyond  the  sample,  it  was 
interpreted  as  an  indication  that  the  users  in  the  study  did  shift 
their  perceived  information  requirements  fairly  quickly  after  MIS 
project  implementation.   While  users  may  have  scored  a  project 
very  high  on  how  well  it  served  their  information  needs  when 
first  implemented,  they  tended  to  score  present  satisfaction  at 
a  lower  level.   This  interpretation  would  seem  to  be  a  good 
argument  for  the  evolutionary  approach  to  MIS  projects  since 
such  projects  have  been  defined  as  management-decision  oriented. 

Finally,  with  respect  to  user  satisfaction,  Appendix  G 
contains  comments  by  users  relative  to  what  they  believed  should 
have  been  done  to  improve  the  projects  in  the  study,  and  what 
they  view  present  shortcomings  to  be.   These  comments  are  not 
direct  quotes;  rather,  they  represent  the  notes  taken  by  the 
researcher  as  the  users  responded  to  an  open-ended  question  in 
the  questionnaire  (Appendix  B,  page  222). 

Computer  Operations  Problems  (Var.  59).   As  reflected 
in  Table  23,  the  measure  of  computer  operations  success  used  in 
the  study  was  related  to  a  secondary  measure  of  such  success, 
namely,  the  operations  cost  of  a  project.   This  relationship 
indicates  that  projects  which  were  implemented  most  smoothly 
by  computer  operations  were  also  the  most  successful  in  terms 
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of  computer  operating  costs.   This  relationship  is  not 
surprising,  and  would  appear  to  be  the  underlying 
explanation  of  the  relationship  between  user  satisfaction 
and  operations  cost  success  shown  in  Table  22.   It  will  be 
recalled  that  the  relationship  of  user  satisfaction  to 
computer  operations  success  was  discussed  earlier. 

TABLE  23 
Computer  Operations  Problems (Var.  59) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 
Computer  Operations  Cost       60     .420    2.078    .038 


Further  Comments  Concerning  the  Hypothesis  Variables 

Tables  24  through  37  contain  relationships  of  the 
hypothesis  variables  to  other  non-criterion  variables  for 
which  data  were  collected  in  the  study.   Each  relationship 
tabled  will  not  be  dealt  with  in  detail.   Rather,  patterns 
that  were  detected,  and  findings  of  particular  interest,  will 
be  dealt  with.   The  discussions  which  follow  are  oriented 
around  the  hypothesis  variables,  and  are  presented  in  the 
same  sequence  in  which  these  variables  appeared  in  the 
earlier  chapter  dealing  with  the  testing  of  the  hypotheses. 

User  Participation-User  (Var.  46).   User  participation 
in  project  development,  as  perceived  by  user  personnel,  was 
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related  to  more  separate  variables  than  any  other  variable 
in  the  study,  as  shown  in  Table  24.   However,  several  of  these 
relationships  tended  to  cluster  together,  revealing  what 
appeared  to  be  more  basic  underlying  relationships. 

TABLE  24 

User  Participation-User   (Var.   46) 

(Two-tailed  significance  at  or  beyond    .10  only) 

Var.  Tau  Z(s)  P(Z) 


User  Satisfaction-PL 

57 

.644 

3.530 

.0004 

Project  Success -PL 

56 

.553 

2.887 

.004 

Satisfaction  of  Objectives-PL 

55 

.513 

2.767 

.006 

Time  Success -PL 

49 

.506 

2.737 

.006 

Assets 

1 

.417 

1.904 

.058 

Specificity  of  User 

Requirements -PL 

38 

.400 

2.160 

.030 

Measurable  Project  Objectives 

12 

.394 

2.112 

.036 

Complexity 

13 

.369 

2.009 

.046 

User  Participation -PL 

41 

.367 

2.174 

.030 

Specificity  of  User 

Requirements -User 

45 

.356 

2.111 

.034 

Hardware   Investment /Sales 

6 

.309 

1.772 

.076 

Implementation  Problems-User 

53 

.291 

1.701 

.090 

Two  or  More  Years    Systems 

Experience 

27 

B-.353 

1.984 

.048 
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User  participation  was  significantly  related  to 
four  separate  measures  of  how  successful  the  project  leader 
perceived  the  project  to  be  (Variables  49,  55,  56,  and  57 )„ 
The  underlying  relationships  here  would  appear  to  be  the 
relationship  of  user  participation  to  user  satisfaction,  and 
the  relationship  of  user  satisfaction  to  the  project  leader's 
perceptions  of  success.   One  interesting  exception  was  the 
relationship  of  user  participation  to  the  project  leader's 
perception  of  time  success.   It  appeared  that  where  perceived 
user  participation  was  high,  project  leaders  tended  to  discount 
the  actual  time  performance  in  evaluating  time  success.   This, 
in  turn,  implies  that  project  leaders  were  more  oriented  to 
how  the  users  viewed  the  projects  than  to  the  actual  time 
spent  compared  to  time  estimated.   This  observation  will  be 
dealt  with  in  more  detail  further  on. 

Those  users  in  organizations  which  were  large  in 
terms  of  assets,  and  which  had  high  computer  hardware  to  sales 
ratios,  felt  that  they  participated  more  in  project  development 
efforts.   This  probably  reflected  a  stage  of  development  in 
computer  usage  which  the  larger,  more  heavily  computer  oriented 
organizations  had  reached. 

With  respect  to  the  reported  specificity  of  project 
requirements,  and  the  reported  clarity  of  project  objectives, 
user  participation  was  perceived  to  be  higher  when  the  measures 
of  these  variables  were  higher.   It  was  difficult  to  determine 
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precisely  what  was  behind  these  relationships,  but  it  seems 
that  users  felt  they  participated  to  a  greater  extent  where 
projects  were  well  defined  and  objectives  were  clearly  stated 
from  the  beginning.   The  specificity  and  clarity  of  objectives 
would  indicate  that  users  knew  what  they  wanted  and  were  in  a 
position  to  actively  work  with  the  systems  staffs  in  getting 
it.   Where  objectives  were  vague,  and  the  users  weren't  really 
certain  what  they  were  after,  the  information  systems  staffs 
picked  up  the  ball  and  did  more  or  less  as  they  saw  fit. 
This  is  not  to  impugn  the  Information  systems  staffs.   Rather, 
given  somewhat  vague  requirements,  they  had  to  fill  in  the 
gaps  themselves  to  get  something  running,  relegating  the  users 
to  question-answerers  when  the  systems  people  got  stuck.   The 
important  implication  in  these  conclusions  would  seem  to  be  the 
necessity  for  clear  objectives  and  specific  information  require- 
ments from  users  if  the  users  want  to  be  heavily  involved  in 
the  development  effort,  thus  providing  greater  assurance  that 
the  project  results  will  satisfy  their  information  needs. 

Perceived  user  participation  was  also  related  to 
having  fewer  implementation  problems.   This  logically  would 
follow  from  closer  user  involvement  during  project  development, 
thereby  providing  the  users  with  a  more  precise  understanding 
of  what  was  to  be  implemented  and  possible  attendant  problems. 

Finally,  one  relationship  shown  in  Table  24  which 
may  appear  somewhat  odd  was  the  one  between  perceived  user 
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participation  and  the  systems  experience  of  the  project  staff. 
Being  a  negative  relationship,  one  might  conclude  that  staff 
personnel  with  greater  systems  experience  did  not  permit  much 
user  participation.  What  actually  seems  to  have  been  the  case, 
however,  was  that  user  members  of  project  teams,  having  little, 
If  any,  systems  experience,  pulled  the  systems  experience 
measure  down.   Since  having  user  personnel  on  project  teams 
generally  contributed  to  higher  perceptions  of  user  involve- 
ment, the  negative  relationship  between  participation  and 
systems  experience  resulted. 

Measurable  Project  Objectives  (Var.  12).   There  were 
two  main  clusters  of  variables  related  to  the  reported  clarity 
of  project  objectives.   The  first  cluster,  consisting  of 
Variables  1,  2,  3,  and  6  in  Table  25,  represent  organization 
and  computer  installation  size.   The  larger  organizations,  and 
those  most  heavily  committed  to  computerized  information  process- 
ing, tended  to  have  the  most  specific  and  measurable  project 
objectives.   This,  again,  would  seem  to  indicate  a  more  advanced 
stage  of  computer  usage. 

The  second  cluster  of  variables  (14,  27,  39,  41,  and 
46)  seems  to  represent  the  effects  of  clear  project  objectives 
on  other  aspects  of  project  development.   Where  project  object- 
ives were  reported  to  have  been  clearly  stated  from  the  beginning 
of  a  project,  the  users  appeared  to  be  more  involved  in  all 
subsequent  project  efforts,  a  point  made  earlier.   Project  teams 
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were  more  common,  top  level  user  management  took  a  more  active 
Interest  in  what  was  happening,  and  user  participation,  as 
perceived  by  both  the  project  leaders  and  users,  was  higher,, 
All  of  these  factors  combined  to  present  a  picture  of  users 
knowing  what  they  wanted  and  actively  participating  with  the 
systems  staff  in  getting  it.   In  addition,  where  project  object- 
ives were  reported  to  have  been  clearly  stated  in  the  beginning, 
there  were  fewer  course  changes  during  project  development,  as 
indicated  by  the  negative  relationship  between  objectives   and 
the  measure  of  project  change  requests  honored  by  the  project 
staff  (Var.  40). 

TABLE  25 
Measurable  Project  Objectives  (Var.  12) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 
Project  Team 
Assets  (14  projects) 
Employees 
Number  on  Project 
User  Participation-User 
Sales  (18  projects) 
User  Management  Interest 
User  Participation-PL 
Hardware  Investment/Sales 

Two  or  More  Years  Systems 
Experience 

Changed  as  Requested 


14 

.580 

2.320 

.020 

1 

.574 

2.507 

.012 

3 

.437 

2.321 

.020 

17 

.400 

2.138 

.032 

46 

.394 

2.112 

.036 

2 

.386 

1.921 

.054 

39 

.347 

1.806 

.072 

41 

.344 

1.830 

.068 

6 

.312 

1.660 

.096 

27 

-.400 

2.146 

.032 

40 

-.405 

1.922 

.054 
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An  Interesting  exception  to  the  above  general 
conclusions  was  the  case  where  project  objectives  were  scored 
high,  but  the  specific  user  requirements  were  scored  low.  There 
were  three  projects  in  this  category.  For  these  three  projects, 
the  original  objectives  were  reported  to  have  been  quantified 
and  clear.   However,  while  the  users  apparently  knew  what  they 
wanted  to  achieve,  they  didn't  appear  able  to  define  the 
information  content  or  format  they  needed  to  achieve  it.   In 
essence,  project  development  seems  to  have  begun  before  the 
users  had  analyzed  their  own  information  requirements  well 
enough  to  define  them  clearly  to  the  systems  staff.   The  result 
in  all  three  cases  was  considerable  frustration  among  all 
concerned,  and  very  poor  time  performance  on  the  projects 
involved.   The  point  of  all  this  is  that  specific  user  require- 
ments did  not  automatically  follow  from  well  specified  object- 
ives, and  where  they  did  not,  considerable  difficulties  were 
encountered  in  project  development.   The  fact  that  users  were 
able  to  state  clearly  what  they  wanted  to  achieve  did  not  mean 
that  they  understood  their  own  information-decision  environments 
well  enough  to  plot  a  course  to  get  there. 

Project  Team  (Var.  14).   It  was  expected  that  the  use 
of  a  project  team,  composed  of  user  representatives  and  informa- 
tion systems  staff,  would  enhance  the  level  of  user  participation 
in  project  development.   This  expectation  was  confirmed  when 
user  participation  was  seen  through  the  eyes  of  the  project  leader, 


-125- 


Where  there  were  teams,  consisting  in  part  of  user  personnel, 
project  leaders  felt  that  objectives  were  clear,  user  require- 
ments were  more  specifically  stated,  and  user  participation 
was  quite  high  (see  Table  26). 

TABLE  26 
Project  Team  (Var.  14) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(2) 

Specificity  of  User 

Requirements  38 

User  Participation-PL         41 

Measurable  Project  Objectives   12 

Number  on  Project  17 

Two  or  More  Years  Systems 

Experience  27    -.760    3.018    .004 

However,  the  measure  of  user  participation  representing  the 
users'  perceptions  was  not  significantly  related  to  the  existence 
of  a  project  team.   This  surprising  finding  for  the  whole  sample 
led  to  a  detailed  review  of  each  project  in  an  effort  to  explain 
the  divergence  of  the  users'  and  the  project  leader's  views  of 
ostensibly  the  same  factor,  user  participation.   This  detailed 
review  revealed  the  following: 

1.  The  three  highest  ranked  projects  on  user 

participation  (users'  view)  were  all 

developed  by  project  teams. 


.750 

3.032 

.0027 

.660 

2.603 

.009 

.580 

2.320 

.020 

.500 

1.964 

.050 
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2.  The  two  lowest  ranked  projects  on  user 
participation  (users'  view)  were  not 
developed  by  project  teams. 

3.  Four  projects  where  teams  were  used,  but 
which  had  low  ranks  on  user  participation 
(users'  view),  seemed  to  reflect  consider- 
able disagreement  among  individual  user 
respondents  on  how  much  they,  as  users, 
participated  in  project  development. 

To  explore  item  3  above  further,  eighteen  of  the  questionnaires 
were  analyzed  in  terms  of  individual  user  responses  to  the 
four  questions  that  were  aggregated  in  arriving  at  the  user 
participation  scores  for  these  projects.   Only  eighteen  projects 
were  analyzed,  since  for  two  projects  there  was  only  one  user, 
and  no  disagreement  on  the  separate  questions  was  possible.   For 
each  of  the  eighteen  projects  analyzed,  an  "index  of  disagreement" 
was  computed  as  follows: 

For  each  of  the  four  questions  comprising  the 
composite  variable,  user  participation,  the 
difference  between  two  separate  user  responses 
was  determined  by  merely  subtracting  the  low 
response  from  the  high  response  on  the  fifty 
point  scale.   These  four  differences  were 
then  summed  together  to  get  an  index  of 
disagreement  for  the  project.   For  example, 
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if  one  user  responded  to  the  question 
"What  was  the  degree  of  your  organization's 
active  participation  throughout  the  evolu- 
tion of  this  project?"  with  a  scale  value 
of  forty,  and  the  other  user  responded 
with  a  scale  value  of  twenty,  the  difference 
for  that  one  question  was  twenty.   That 
difference  was  then  added  to  the  remaining 
three  differences  computed  in  the  same 
manner  to  arrive  at  the  index  value  for  the 
project. 

The  mean  value  of  the  index  of  disagreement  for  all 
eighteen  projects  was  45.6.  When  the  index  of  disagreement 
value  for  each  of  the  four  projects  mentioned  in  item  3  above 
was  compared  to  this  mean  index  value,  it  was  found  that  these 
four  projects  had  the  highest  index  of  disagreement  scores  in 
the  sample,  with  two  at  70,  one  at  100,  and  one  at  120.  Further, 
all  four  of  these  projects  had  one  thing  in  common:   each  was  a 
case  where  one  of  the  user  personnel  interviewed  had  actually 
worked  on  the  project  team  and  the  other  user  had  not.   As 
would  be  expected,  the  individuals  who  had  actually  been  on 
project  teams  scored  user  participation  consistently  higher 
than  did  the  users  who  had  not  been  on  the  project  teams. 

What  the  above  findings  would  seem  to  Indicate  is 
that  users  disagreed  among  themselves  concerning  how  much  they 
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participated  in  a  project.  While  the  individuals  who  were 
actually  on  the  team  felt  that  participation  was  high,  other 
users  often  felt  they  had  very  little  to  do  with  the  project. 
Where  the  objective  of  a  project  was  to  provide  management 
information  to  several  people  in  a  user  area,  in  fact,  only 
those  who  actually  were  on  the  project  team  ended  up  getting 
what  they  wanted.   Thus,  while  having  a  project  team  did  seem 
to  facilitate  users  feeling  they  participated  to  a  larger 
degree,  a  team  did  not  assure  such  feelings  of  participation. 
This  situation  was  particularly  obvious  where  the  user  represen- 
tatives on  a  team  were  staff  personnel  from  the  user  area,  but 
the  actual  recipients  of  the  results  of  the  project  were  other 
individuals  who  had  not  been  very  much  involved  in  project 
development. 

With  respect  to  the  perceptions  of  project  leaders 
concerning  user  participation,  no  distinctions  were  apparently 
made  among  the  kinds  of  users  who  were  on  project  teams.   The 
mere  presence  of  user  representatives  led  to  perceptions  of 
high  user  participation,  thus  the  high  relationship  between 
Variables  14  and  41  in  Table  26. 

Another  finding  of  interest  relative  to  the  use  of 
project  teams,  was  that  user  satisfaction  with  results  was 
lower  where  the  user  members  of  project  teams  were  not  the 
ones  responsible  for  implementing  the  completed  projects, 
nor  for  maintaining  on-going  liaison  with  the  information 
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sys terns  staffs.   There  were  two  projects  which  fell  in  this 
category.  Whereas  the  mean  value  of  user  satisfaction  (Var.  54) 
for  all  projects  in  the  sample  was  175,  the  mean  value  of  user 
satisfaction  for  the  two  projects  cited  was  122.5.   Both  of  these 
projects  were  in  the  bottom  half  of  the  user  satisfaction  rank- 
ing, with  ranks  of  12  and  15. 

Finally,  the  fact  that  user  representatives  were 
full-time  or  part-time  on  project  teams  seems  to  have  made  no 
difference  on  any  criterion.   Having  part-time  or  full-time 
user  members  on  a  project  team  appeared  to  depend  more  on  the 
nature  of  the  project  and  the  structure  of  the  organization 
than  anything  else. 

As  would  be  expected,  Table  26  shows  that  there 
tended  to  be  more  people  involved  in  projects  where  teams  were 
used.   Also,  the  level  of  systems  experience  was  lower  where 
teams  were  used,  since  users,  who  had  little  or  no  systems 
experience,  pulled  the  measure  of  systems  experience  down. 

Level  of  Information  Systems  Manager  (Var.  9).   For 
the  seven  organizations  in  the  sample  which  provided  asset 
data,  the  larger  organizations,  in  terms  of  assets,  tended  to 
place  the  top  information  systems  executive  closer  to  the  top 
operating  executive  of  the  organization.   This  is  reflected  in 
Table  27  as  a  negative  relationship,  since  a  lower  value  for 
Variable  9  meant  fewer  hierarchical  levels  between  the  informa- 
tion systems  executive  and  the  top  operating  executive. 
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TABLE  27 
Level  of  Information  Systems  Manager  (Var.  9) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     S*     P(s) 
Assets   (N=7)  1    -.796    -13.0    .07 


*  Since  asset  data  were  available  for  only  7  organizations,  the 
normal  deviate  was  replaced  by  the  actual  computed  value  of  S. 
The  probability  level  was  determined  from  Sillitto  (1947). 

Two  or  More  Years  Systems  Experience  (Var.  27).  All 
of  the  relationships  shown  in  Table  28  would  seem  to  reflect, 
essentially,  the  degree  of  user  involvement  in  the  projects  in 
the  study.  There  are  no  relationships  there  which  are  not 
consistent  with  the  discussions  already  presented  for  other 
variables.  Clear  objectives,  specific  user  requirements, 
user  participation,  the  number  of  people  on  the  project,  and 
the  use  of  a  project  team  have  all  been  shown  to  be  related 
to  each  other.  Since  what  was  posited  as  underlying  all  of 
these  relationships  was  user  involvement,  it  is  not  surprising 
that  all  of  the  variables  shown  in  Table  28  were  negatively 
related  to  the  systems  experience  measure,  inasmuch  as  that 
measure  was  lower  the  more  users  were  directly  involved  in 
project  development. 
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TABLE  28 
Two  or  More  Years  Systems  Experience*  (Var.  27) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 

Project  Team  14    -.760    3.018    .003 

Specificity  of  User 
Requirements -PL 

Measurable  Project  Objectives  12 

Number  on  Project 

User  Participation-PL 

User  Participation-User 

User  Satisfaction-PL 

* Since  all  values  for  tau  are  negative  in  this  table,  the 
entries  are  from  most  negative  (strongest  relationship)  to 
least  negative  (weakest  relationship). 

Documentation  Standards  (Var.  35).   Although  the 
reported  existence  of  formal  documentation  standards  is  shown 
to  be  related  to  three  variables  in  Table  29,  great  caution 
must  be  exercised  in  making  any  generalizations  about  these 
relationships.   Only  four  projects  in  the  study  had  no  documen- 
tation standards,  and  two  of  these  projects  were  in  one 
organization.  This  organization  was  the  smallest  in  the  sample 
in  terms  of  sales,  and  next  to  the  smallest  in  number  of 
employees.  Further,  all  of  the  project  staff  members  for  both 


38 

-.450 

2.453 

.014 

12 

-.400 

2.146 

.032 

17 

-.394 

2.311 

.022 

41 

-.361 

2.141 

.032 

46 

B-.353 

1.984 

.048 

57 

-.319 

1.730 

.084 
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of  these  projects  held  college  degrees.   A  third  project  for 
which  no  formal  documentation  standards  applied  was  In  the 
smallest  organization  in  the  sample  in  terms  of  employees,  and 
this  organization  was  next  to  the  smallest  In  sales.  This 
third  project,  a  sophisticated  modeling  application,  was  also 
developed  by  a  team  composed  entirely  of  college  graduates. 
These  factors,  together  with  the  nature  of  the  tau  statistic 
computation,  explain  the  relationships  found.  Therefore, 
generalizing  in  this  case  would  be  very  hazardous. 

TABLE  29 
Documentation  Standards  (Var.  35) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.  Tau  Z(s)  P(Z) 

Sales  (18  projects)            2  .543  2.559  .010 

Employees                     3  .520  2.149  .032 

College  Degree               30  -.490  2.304  .022 

Project  Control  (Var.  42).  The  primary  conclusion  one 
draws  when  looking  at  the  relationships  between  the  project 
control  measure  and  other  variables  in  the  study  is  that 
reported  project  control  efforts  were,  essentially,  ineffective, 
perhaps  even  dysfunctional.   It  has  already  been  shown  in  Tables 
16-19  that  reported  project  control  efforts  were  not  related  to 
any  criterion  of  success.  This  would  seem  to  indicate  that 
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project  status  information,  which  was  ostensibly  collected  for 
control  purposes,  was  not,  in  fact,  used  to  exercise  control, 
at  least  over  the  project  itself.   It  is  worth  noting  that  of 
the  twenty  projects  in  the  study,  in  only  one  case  did 
project  status  reporting  result  in  additional  resources  being 
committed  to  the  project.   That  one  project  was  completed  on 
time.   In  other  projects,  however,  it  appeared  that  project 
status  information  was  collected  as  "nice  to  know",  or,  at 
best,  was  used  to  keep  others  outside  the  information  systems 
function  posted  on  when  they  would  be  affected  by  conversion 
or  implementation  activities.  This  is  not  to  say  that  such 
overall  organizational  awareness  is  not  desirable  nor  neaessary. 
But  awareness  and  positive  steps  to  influence  project  develop- 
ment rate  are  not  synonymous. 

Analysis  of  Table  30  reveals  some  further  interesting 
aspects  of  project  control.   Although  reported  project  control 
was  not  related  to  time  success,  it  was  quite  strongly  related 
to  the  project  leader's  perception  of  time  success.  What  this 
relationship  seems  to  reflect  was  the  tendency  of  the  project 
leader  to  feel  he  had  done  his  best  where  he  was  constantly 
maintaining  close  tabs  on  project  status.   He  didn't  feel  the 
project  had  really  been  so  bad  in  terms  of  time  success,  and 
tended  to  blame  overruns  on  poor  estimates,  although  in  only 
one  case  did  a  project  leader  feel  the  original  time  estimate 
for  the  project  had  been  unrealistic'.   In  fact,  two  projects, 
which  exceeded  time  estimates  by  6007.  and  9007.,  were  rated 
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average  In  terms  of  time  success,  and  three  projects,  which 
exceeded  time  estimates  by  135%,  146%  and  180%,  were  all  rated 
above  average  on  time  success. 

TABLE  30 
Project  Control  (Var.  42) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.  Tau  Z(s)     P(Z) 

Time  Success -PL               49  .540  2.861  .004 

Originator                   11  -.322  1.722  .084 

Quality  of  Documentation       36  B-.336  1.650  .100 

Changed  as  Requested          40  -.345  1.683  .092 

It  appeared  that  project  leaders  felt  an  implicit 
pressure  from  tight  project  reporting  requirements,  but  they 
felt  helpless  in  doing  anything  about  improving  poor  perform- 
ance beyond  cutting  corners  in  such  areas  as  documentation  and 
preparation  for  implementation.   The  negative  relationship 
between  reported  project  control  efforts  and  the  perceived 
quality  of  documentation  would  seem  to  lend  support  to  this 
notion,  as  would  the  negative  relationship  between  reported 
project  control  and  perceived  implementation  problems.   This 
latter  relationship  is  not  tabled,  as  it  was  not  significant 
at  the  .10  level,  but  it  was  fairly  strong,  with  a  tau  of  -.307 
and  a  normal  deviate  of  1.638. 
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Another  Indication  of  the  pressure  felt  by  the 
project  leader  from  project  control  schemes  was  the  reported 
reluctance  to  make  changes  as  development  progressed.   This 
was  not  necessarily  a  bad  thing,  but  the  pressure  to  get  done 
as  soon  as  possible  seems  to  have  prevented  changes  needed  to 
provide  users  with  more  acceptable  outputs  in  some  cases. 
Although  the  reluctance  to  make  desired  changes  did  not  seem 
to  contribute  to  being  on  time  with  projects,  it  probably  did 
prevent  several  of  the  projects  from  being  more  over  estimate 
than  they  already  were. 

Finally,  as  shown  in  Table  30,  those  projects 
initiated  by  the  information  systems  staffs  appeared  to  be 
subject  to  more  stringent  control  requirements  than  those 
initiated  by  users.   It  would  seem  that  the  planning  process 
for  project  development  was  more  explicit  when  a  project  came 
from  the  information  systems  staff,  resulting  in  more  specific 
provisions  for  control  of  the  project  during  its  development. 
This  observation  opens  up  some  very  interesting  possibilities 
with  respect  to  kinds  of  information  systems  projects.  These 
possibilities  will  be  dealt  with  in  Chapter  VIII. 

Turnover  (Var.  29).   It  was  pointed  out  earlier  that 
the  turnover  of  project  personnel  could  not  be  fairly  evaluated 
because  only  three  projects  in  the  study  experienced  any  turn- 
over at  all.   The  same  caution  must  be  observed  in  considering 
the  relationship  of  turnover  to  implementation  problems,  as 
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vlewed  by  the  project  leader,  shown  In  Table  31.  While  the 
turnover  that  did  occur  on  projects  in  the  sample  appeared 
to  create  real  difficulties  for  the  project  leaders  of  those 
projects,  it  would  be  risky  to  generalize  beyond  those  projects, 
The  discussion  of  just  what  implementation  problems  seemed  to 
consist  of  in  the  eyes  of  project  leaders  will  be  dealt  with  in 
the  next  chapter. 

TABLE  31 
Turnover  (Var.  29) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 
Implementation  Problems -PL     58    -.322    2.327    .020 

Originator  (Var.  11).  The  relationship  of  reported 
project  control  efforts  to  origin  of  a  project  has  already 
been  discussed.   The  relationship  of  project  origin  to  implemen- 
tation problems,  as  viewed  by  the  user,  shown  in  Table  32, 
indicates  that  users  felt  they  had  less  difficulty  implementing 
projects  they  initiated  than  those  initiated  by  higher  level 
management  or  information  systems  staffs.  This  finding  is  not 
at  all  surprising,  and  fits  the  patterns  of  relationships 
already  discussed. 

One  finding  that  was  surprising,  however,  was  the 
absence  of  a  significant  relationship  between  origin  of  a 
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project  and  the  users'  perception  of  participation  In  project 
development.  Although  these  two  variables  were  related,  with 
a  tau  of  .29  and  a  normal  deviate  of  1.493,  this  relationship 
failed  to  reach  significance  at  the  .10  level. 

TABLE  32 
Originator  (Var.  11) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     2(s)     P(Z) 
Implementation  Problems-User   53     .330    1.709    .088 
Project  Control  42    -.322    1.722    .084 

A  closer  Investigation  of  the  sample  data  revealed 
what  would  appear  to  explain  this  situation.  Two  of  the  thirteen 
projects  Initiated  by  users  had  very  low  values  for  perceived 
user  participation,  120  In  both  cases.  This  value  was  well 
below  the  overall  mean  for  user  participation,  152.6,  and  was 
far  below  the  mean  user  participation  value  of  164  for  those 
thirteen  projects  which  were  originated  by  users.   The  two 
projects  In  question,  although  originated  by  users,  were 
developed,  for  the  most  part,  without  further  substsntlve  user 
participation.   In  one  case,  this  appeared  to  be  the  result  of 
the  users  not  wanting  to  be  bothered;  and  In  the  other  case,  the 
project  leader  simply  did  not  work  with  the  users,  choosing  to 
proceed  on  his  own.   It  Is  worth  noting  that  these  two  projects 
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were  both  very  low  In  the  ranking  of  user  satisfaction,  with 
one  project  ranked  17,  and  the  other  project  ranked  18. 

Based  on  the  above  findings,  It  appears  that  the 
fact  a  user  originated  a  project  did  not  guarantee  his  partici- 
pation In  Its  development.   However,  the  results  were  not  very 
satisfying  to  the  user  where  participation  did  not  follow 
from  project  Initiation. 

Centralization  of  MIS  Activities  (Var.  8).  This 
variable  was  not  adequately  tested  In  the  present  study. 

Two  or  More  Years  Organization  Experience  (Var.  34). 
There  were  few  findings  with  respect  to  organization  experience 
of  the  project  staff  which  were  of  any  significance.  As  might 
be  expected,  Table  33  shows  that  larger  organizations,  in 
terms  of  numbers  of  employees,  had  project  staffs  with  less 
organization  experience.  This  might  be  Interpreted  as  an 
Indication  that  more  fluid  personnel  situations  existed  in 
larger  organizations.  Table  33  also  shows  that  the  education 
level  of  the  project  staff  was  related  to  organization  experience, 
This  finding  would  seem  to  run  counter  to  the  popular  notion 
that  people  involved  in  systems  work,  and  who  are  highly 
educated,  have  high  job  turnover  rates.  At  least  for  this 
sample,  those  who  were  more  highly  educated  tended  to  have 
been  In  their  current  organizations  for  at  least  two  years. 
Caution  must  be  exercised  In  making  too  much  of  the  above 
finding  concerning  turnover  and  education,  however.  The 
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measures  used  were  not  direct  individual  measures  of  job 
longevity  and  education  level.  Rather,  they  were  mean  values 
and  proportions,  which  were  felt  to  be  the  best  types  of 
measures  for  the  primary  tasks  in  this  study,  but  which  were 
not  the  best  measures  for  an  analysis  of  turnover  as  it 
relates  to  education. 

TABLE  33 
Two  or  More  Years  Organization  Experience  (Var.  34) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 


College  Degree 

30 

B.375 

2.132 

.034 

Mean  Years  Formal  Education 

32 

.372 

2.046 

.040 

Employees 

3 

-.411 

2.401 

.016 

High  Level  Programming  Language  (Var.  43).   The 
relationship  between  the  use  of  a  higher  level  programming 
language  and  the  quality  of  project  documentation,  shown  in 
Table  34,  reflects  primarily  the  views  of  project  leaders 
that  the  use  of  COBOL  resulted  in  good  program  documentation. 
Since  the  perceived  quality  of  program  documentation  appeared 
to  be  a  key  element  in  evaluating  overall  project  documentation, 
this  finding  is  not  surprising. 
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TABLE  34 
High  Level  Programming  Language  (Var.  43) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 
Quality  of  Documentation       36     .300    1.780    .076 

Mean  Years  Formal  Education  (Var.  32).   The  relationship 
found  between  project  personnel  education  levels  and  organization 
experience  has  been  dealt  with  to  some  extent  in  the  discussion 
of  the  organization  experience  variable.  As  shown  in  Table  35, 
there  were  four  other  variables  significantly  related  to  project 
staff  education  besides  organization  experience.  Looking  at  all 
five  of  these  variables  together,  one  finds  a  pattern  that  seems 
to  explain  their  relationships  to  education  levels  of  project 
staffs.  The  smaller  organizations  in  the  sample,  in  terms  of 
numbers  of  employees,  tended  to  have  smaller  project  staffs  whose 
members  had  high  education  levels,  and  who  had  been  employed  in 
their  respective  organizations  for  over  two  years.  Further, 
smaller  project  staffs  tended  to  have  combination  analyst/ 
programmers.  Finally,  users  generally  perceived  implementation 
problems  to  be  the  least  where  combination  analyst/programmers 
developed  projects  (see  Table  36).   The  above  pattern  was  partic- 
ularly pronounced  where  applications  dealing  with  mathematical 
models  were  involved.   This  does  not  mean  that  smaller  organizations 
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had  the  most  successful  projects  or  the  projects  with  the  least 
Implementation  problems.  Rather,  the  Interrelations  of  the 
several  variables  discussed  provide  a  possible  explanation 
for  findings  relative  to  the  education  level  of  project  personnel, 

TABLE  35 
Mean  Years  Formal  Education  (Var.  32) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 

Combination  Analyst/Programmer  23     .440    1.668    .096 

Two  or  More  Years  Organization 
Experience  34 

Implementation  Problems-User   53 

Number  on  Project  17 

Employees  3 

TABLE  36 

Combination  Analyst/Programmer  (Var.  23) 
(Two-tailed  significance  at  or  beyond  .10  only) 


.372 

2.046 

.040 

.336 

1.884 

.060 

-.330 

1.821 

.068 

-.378 

2.073 

.038 

Var. 

Tau 

Z(s) 

P(Z) 

College  Degree 

30 

.590 

2.238 

.026 

Implementation  Problems -User 

53 

.530 

2.029 

.042 

Mean  Years  Formal  Education 

32 

.440 

1.668 

.096 

Man  Months 

22 

-.500 

1.868 

.062 

Number  on  Project 

17 

-.610 

2.306 

.022 
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Combinatlon  Analyst/Programmer  (Var.  23).   The 
finding  in  the  study  that  project  staffs  comprised  of 
combination  analyst /programmers  had  higher  levels  of  education 
has  already  been  mentioned  in  the  preceding  discussion  of  the 
primary  measure  of  education  level.   Table  36  also  shows  that 
there  were  higher  proportions  of  college  degrees  on  project 
staffs  composed  of  combination  analyst/programmers.   This  is 
merely  one  more  way  of  looking  at  the  same  basic  relationship. 

The  tendency  for  users  to  feel  implementation 
problems  were  less  with  combination  analyst/programmers  has 
also  been  cited  on  several  occasions.  As  was  pointed  out 
before,  users  seemed  to  feel  frustrated  and  somewhat  dissatisfied 
with  implementation  efforts  where  they  had  to  deal  with  differ- 
ent people  from  the  information  systems  staff,  depending  on  the 
nature  of  the  problems  that  came  up.   The  ability  to  deal  with 
one  person  who  understood  all  aspects  of  the  project,  and  who 
could  respond  to  problems  with  action,  was  often  mentioned  by 
users  as  a  very  strong  factor  in  their  satisfaction  with  project 
results.   It  appears  that  combination  analyst/programmers  were 
better  able  to  meet  users'  desires  in  this  respect  than  were 
separate  analysts  or  programmers. 

Finally,  combination  analyst/programmers  generally 
meant  fewer  people  on  a  project,  and  the  projects  they  worked 
on  took  fewer  man  months.   This  latter  relationship  is  open 
to  interpretation.   It  could  be  that  since  there  were  fewer 
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people  working  on  projects  where  combination  analyst/programmers 
were  used,  the  man  months  being  less  just  reflected  that  under- 
lying relationship.   On  the  other  hand,  it  could  be  that  there 
was  less  lost  time  and  motion  where  combination  analyst/ 
programmers  were  used,  thus  requiring  fewer  man  months  to 
complete  a  project.  While  this  latter  argument  is  appealing 
from  a  logical  standpoint,  it  is  hard  to  support  in  view  of 
the  fact  that,  as  shown  earlier  in  the  discussion  of  criteria- 
hypotheses  relationships,  project  staffs  composed  of  combination 
analyst/programmers  performed  rather  poorly  on  the  time  success 
criterion.   Another  possible  explanation  might  be  that  combina- 
tion analyst/programmers  were  used  on  smaller,  less  complex 
projects.  This  position  is  also  difficult  to  defend,  however, 
since  there  was  little  relationship  between  the  measure  of 
complexity  (Var.  13)  and  the  use  of  combination  analyst/programmers. 
Perhaps  the  question  could  have  been  resolved  had  there  been  a 
comparable,  objective  measure  of  project  complexity  which  was 
applied  equally  to  all  projects  by  the  researcher.   In  the 
absence  of  such  a  measure,  no  satisfactory  explanation  could 
be  found  by  the  researcher  for  the  relationship  between  using 
combination  analyst/programmers  and  man  months  spent  on  a 
project.   The  whole  question  of  combination  versus  separate 
analyst/programmers  would  seem  to  be  a  very  fruitful  one  for 
further  research. 
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System3  Staff /Total  Employees  (Var.  61).   The 
measure  of  relative  size  of  sample  organizations'  systems 
staffs  was  not  significantly  related  to  any  other  independent 
variable  with  which  it  was  cross -tabula ted. 

Hardware  Investment /Sales  (Var.  6).   Organizations 
in  the  sample  with  the  highest  asset  valuations  also  tended 
to  have  the  highest  ratios  of  computing  hardware  investment 
to  sales  for  the  past  fiscal  year.  This  relationship  was 
computed  for  only  the  seven  organizations  which  provided 
asset  data,  however  (see  Table  37). 

TABLE  37 
Hardware  Investment/Sales  (Var.  6) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.  Tau  Z(s)     P(Z) 

Assets  (Ns7)                  1  .784  S«16.0*  .02 

Measurable  Project  Objectives   12  .312  1.660  .096 

User  Participation -User        46  .309  1.772  .076 


*  Since  asset  data  were  available  for  only  7  organizations, 

the  normal  deviate  was  replaced  by  the  actual  computed  value 

of  S.   The  probability  level  was  determined  from  Kendall  (1962). 


The  hardware  investment/sales  ratio  was  also  related 
to  the  reported  clarity  of  project  objectives,  and  to  user 
perceptions  of  participation  in  project  development.   It  has 
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already  been  shown  that  perceived  user  participation  was  related 
to  reported  clarity  of  project  objectives  (Table  24).   As  was 
pointed  out  in  the  discussion  of  relationships  shown  in  Table 
24,  a  high  hardware/sales  ratio  was  construed  to  represent  a 
strong  commitment  by  an  organization  to  computerized  processing 
of  information  for  decision  making.   In  such  an  environment, 
it  appears  that  clear  objectives  were  required  before  a  project 
was  initiated,  and  users  were  more  involved  in  the  planning  for 
their  own  information  systems. 


' 
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CHAPTER  VII 
OTHER  RELATIONSHIPS  OF  INTEREST 

Although  the  author's  primary  Intent  In  this  thesis 
has  been  to  report  the  results  of  conducting  tests  of  the 
hypotheses  posited  for  the  study,  the  nature  of  the  data 
collected  has  made  It  possible  to  Investigate  and  report 
certain  other  relationships  of  relevance  and  Interest.  Some 
of  these  "other"  relationships  have  already  been  discussed 
In  the  last  two  sections  of  Chapter  VI.  This  chapter  focuses 
mainly  on  those  findings  which  provide  Insight  Into  the  ways 
project  leaders  viewed  their  respective  projects. 

Project  Leaders'  Views  on  Project  Success 
Tables  38-41  provide  a  picture  of  the  way  project 
leaders  viewed  project  success  In  terms  of  time,  cost,  user 
satisfaction,  and  for  the  project  overall.  No  attempt  has  been 
made  to  discuss  every  relationship  shown  in  all  four  tables. 
Rather,  what  appeared  to  be  meaningful  patterns  of  relationships 
have  been  dealt  with. 

First,  although  project  leaders'  evaluations  of 
time  and  cost  success  were  related  to  the  actual  measures  of 
time  success  and  cost  success  respectively,  the  most  prominent 
factor  underlying  the  overall  evaluations  of  success  appeared 
to  be  how  satisfied  the  project  leaders  perceived  the  users 
were  with  the  project  results.  To  support  this  contention, 
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consider  the  results  In  Tables  40  and  41.  When  asked  to  rate  their 
projects  on  how  successful  they  were  overall,  the  project  leaders' 
responses  were  related  to  a  cluster  of  other  variables  which  repre- 
sented, essentially,  user  satisfaction.  The  only  exception  was 
the  relationship  between  overall  project  success  and  time  success. 

TABLE  38 
Time  Success -PL  (Var.  49) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 


Pro j act  Control 

42 

.540 

2.861 

.004 

User  Participation -User 

46 

.506 

2.737 

.006 

Project  Success -PL 

56 

.487 

2.578 

.010 

Cost  Success -PL 

52 

B.447 

2.335 

.020 

Specificity  of  User 
Requirements -User 

45 

.394 

2.114 

.034 

User  Sat is fact ion -PL 

57 

B.387 

2.006 

.044 

Elapsed  Months 

21 

-.369 

1.963 

.050 

Actual  Time/Estimated  Time 

48 

-.437 

2.328 

.020 

TABLE  39 

Cost  Success -PL  (Var.  52) 

(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau  Z(s)     P(Z) 

Time  Success-PL  49  B.447  2.335    .020 

Number  on  Project  17  -.319  1.707    .088 

Actual  Cost/Estimated  Cost     51  -.644  3.444    .0006 
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TABLE  40 

Project  Success -PL  (Var.  56) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 

User  Satisfaction-PL 

User  Satisfaction-User 

Implementation  Problems -User 

User  Participation-User 

Time  Success -PL 

Satisfaction  of  Objectives-PL     55 

Specificity  of  User 

Requirements -PL  38  .367  1.937  .052 

TABLE  41 
User  Satisfaction-PL  (Var.   47) 
(Two-tailed  significance  at  or  beyond   .10  only) 

Var.  Tau  Z(s)  P(Z) 


57 

.713 

3.837 

.0002 

54 

.607 

3.124 

.002 

53 

.553 

2.920 

.004 

46 

.553 

2.887 

.004 

49 

.487 

2.578 

.010 

<  55 

B.407 

1.979 

.048 

Project  Success -PL 

56 

.713 

3.837 

.0002 

User  Sat is fact ion -User 

54 

.656 

3.548 

.0004 

User  Participation -User 

46 

.644 

3.530 

.0004 

Implementation  Problems -User 

53 

.519 

2.871 

.004 

Satisfaction  of  Objectives -PL 

55 

.447 

2.469 

.014 

Specificity  of  User 
Requirements -PL 

38 

B.409 

2.116 

.036 

User  Participation -PL 

41 

.394 

2.135 

.032 

Time  Success -PL 

49 

B.387 

2.006 

.044 

Two  or  More  Years  Systems 
Experience 

27 

-.319 

1.730 

.084 

Man  Months 

22 

-.350 

1.880 

.060 
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While  it  appears  that  project  leaders  did  give  some 
consideration  to  time  success,  as  they  had  evaluated  it,  in 
arriving  at  an  overall  rating  of  project  success,  that  relation- 
ship, too,  may  reflect  a  "user  satisfaction"  bias.   This  last 
statement  follows  from  the  findings  in  Table  38.   Time  success, 
as  viewed  by  the  project  leader,  was  related  to  reported  project 
control,  which  has  already  been  discussed;  to  the  actual  time 
success  measure;  and  to  the  project  leader's  rating  of  cost 
success.   However,  it  was  also  related  to  user  satisfaction, 
as  viewed  by  the  project  leader,  perceived  user  participation, 
and  the  reported  specificity  of  user  requirements.   These 
last  mentioned  relationships  could  reflect  the  tendency  of 
project  leaders  to  view  time  success  in  the  context  of  how 
satisfied  they  felt  users  were  with  project  results.   This 
possibility  is  supported  by  the  fact  that  the  two  lowest 
ranked  projects  (19  and  20)  on  the  user  satisfaction  criterion 
were  rated  at  the  lowest  level  (very  poor)  on  time  success 
(Var.  49)  by  the  project  leaders.   However,  these  two  projects 
were  ranked  13  and  14  on  the  actual  measure  of  time  success 
(Var.  48). 

On  the  other  hand,  it  may  be  that  project  leaders 
did  rate  overall  project  success  with  consideration  given  to 
their  perceptions  of  both  time  success  and  user  satisfaction. 
If  this  was  the  case,  and  the  two  projects  cited  in  the  previous 
paragraph  were  merely  chance  occurrences,  the  user  satisfaction 
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relationshtp  to  perceived  time  success  could  be  explained  by 
the  common  relationship  of  both  of  those  variables  to  overall 
project  success  (Var.  56). 

With  respect  to  the  strong  relationship  between  the 
project  leader's  perceptions  of  time  success  and  cost  success, 
it  would  appear  that  a  project  leader  response  set  was  partially 
responsible  for  this  relationship.   Although,  as  Table  15 
shows,  the  actual  measures  of  time  success  and  cost  success 
were  related,  that  relationship  was  not  significant  at  the 
.10  level.   This  led  to  the  conclusion  that  project  leaders 
tended  to  rate  cost  success  and  time  success  at  about  the 
same  level,  regardless  of  how  similar  the  actual  time 
success  and  cost  success  values  were. 

Perhaps  the  most  important  point  to  be  made  concerning 
the  relationships  shown  in  Tables  40  and  41  is  how  accurately 
the  project  leaders  perceived  user  satisfaction  with  their 
projects.   It  was  evident  that  in  nearly  all  cases,  project 
leaders  were  very  well  attuned  to  how  the  users  viewed  project 
results.   It  was  also  apparent,  as  has  been  discussed  above, 
that  those  user  attitudes  were  key  determinants  in  how 
successful  project  leaders  viewed  their  projects  to  be. 

Different  Views  of  Implementation  Problems 
One  interesting  aspect  of  project  leaders'  perceptions 
concerning  user  satisfaction  was  their  awareness  of  implementa- 
tion problems,  as  viewed  by  the  users.   As  Tables  40  and  41 
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show,  the  users'  evaluation  of  Implementation  problems  was  very 
strongly  related  to  how  well  satisfied  the  project  leader 
perceived  the  users  to  be,  and,  in  turn,  to  how  successful  the 
project  leader  felt  the  overall  project  was.   It  has  already 
been  shown  (Table  22)  that  users  considered  ease  or  difficulty 
of  project  implementation  to  be  a  very  significant  component 
of  overall  satisfaction  with  a  project. 

However,  as  Tables  42  and  43  show,  project  leaders 
viewed  implementation  problems  very  differently  than  users  did. 
Not  only  were  the  two  views  of  implementation  problems  different, 
project  leaders  did  not  even  consider  implementation  problems 
to  be  related  to  any  criterion  of  success. 

Most  of  the  relationships  in  Table  42  have  been 
covered  in  earlier  discussions  of  the  several  variables  shown, 
so  there  is  no  need  to  repeat  what  has  been  said  about  them. 
However,  taking  all  of  the  variables  together,  and  considering 
the  pattern,  provides  insight  into  what  decreased  implementation 
problems  from  the  users'  perspective. 

Implementation  to  users  appears  to  have  meant  getting 
the  project  and  its  outputs  incorporated  into  their  ongoing 
operations  as  smoothly  and  effectively  as  possible.   The  object- 
ive of  the  user  was  to  get  an  information  system  product  which 
aided  in  decision  making,  but  which  also  caused  the  least 
turbulence  in  the  organization.   Achieving  this  objective 
appeared  to  be  tied  to  how  prepared  the  user  was  for  what  he  got. 
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TABLE  42 
Implementation  Problems -User  (Var.  53) 
(Two-tailed  significance  at  or  beyond  .10  only) 


Var. 


Tau 


Z(s)     P(Z) 


User  Satisfaction-User 

54 

.720 

4.192 

.0004 

Project  Success-PL 

56 

.553 

2.920 

.004 

Combination  Analyst /Programmer 

23 

.530 

2.029 

.042 

User  Satisfaction-PL 

57 

.519 

2.871 

.004 

Specificity  of  User 
Requirements -PL 

38 

.419 

2.290 

.021 

Mean  Years  Formal  Education 

32 

.336 

1.884 

.060 

Quality  of  Documentation 

36 

.333 

1.739 

.082 

Originator 

11 

.330 

1.709 

.088 

User  Participation-User 

46 

.291 

1.701 

.090 

Man  Months 

22 

-.383 

2.217 

.026 

TABLE  43 
Implementation  Problems -PL  (Var.  58) 
(Two-tailed  significance  at  or  beyond  .10  only) 


Var. 


Tau 


Z(s)     P<2) 


Quality  of  Documentation 

36 

.533 

2.869 

.004 

Turnover 

29 

-.322 

2.327 

.020 

Complexity 

13 

B-.352 

1.793 

.074 
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When  he  had  originated  a  project,  participated  heavily  in  its 
development,  and  had  good  project  documentation  to  work  with, 
the  user  knew  what  was  coming.   Further,  where  he  was  able 
to  deal  with  one  individual  who  knew  the  whole  picture  when 
problems  did  arise,  the  user  felt  he  was  able  to  get  those 
problems  rectified  quickly. 

Contrast  this  to  how  the  project  leader  viewed  imple- 
mentation problems.   Although,  as  was  mentioned  earlier,  the 
project  leader  was  aware  of  the  users'  view  of  implementation 
problems,  he  did  not  consider  the  same  factors  to  be  the 
components  of  implementation  problems,  with  the  exception  of 
documentation.   The  project  leader  tended  to  view  successful 
implementation  as  getting  the  programs  running,  the  documenta- 
tion finished,  and  the  procedural  flows  correct  so  that  his 
system  worked.   Turnover,  although  very  limited  in  the  sample, 
was  seen  as  standing  in  the  way  of  easy  Implementation,  as  was 
pressure  to  rush  a  project  to  completion.    Reported  complexity 
of  the  project  was  also  negatively  related  to  the  ease  of 
implementation,  but  it  was  difficult  to  determine  which  way 
this  relationship  went.   It  could  be  that  project  leaders 
viewed  as  complex  those  projects  which  were  difficult  to 


Although  not  significant  at  the  .10  level,  implementation 
problems  (Var.  58)  was  negatively  related  to  project  control 
(Var.  42)  with  a  tau  of  -.307  and  a  normal  deviate  of  1.683, 
significant  at  the  .102  level  (two-tailed). 
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imp lement.   Finally,  where  documentation  was  felt  by  project 
leaders  to  be  good,  implementation  problems  were  felt  to  be  less 

Although  not  entirely  accurate,  one  generalization 
which  might  best  describe  the  differing  views  of  implementation 
problems  is  that  project  leaders  tended  to  view  implementation 
up  to  operational  cutover  of  a  project,  while  the  users  looked 
at  implementation  as  what  happened  after  operational  cutover. 

Project  Leader's  Perception  of  User  Participation 

In  general,  the  project  leader  felt  there  was  high 
user  participation  if  a  project  team  consisting  of  some  user 
representatives  developed  the  project,  and  if  he  perceived  top 
level  user  management  had  actively  supported   the  project. 
Most  of  the  relationships  in  Table  45  represent  either  these 
two  basic  variables  or  other  variables  already  shown  to  be 
related  to  them.   Further,  the  project  leader  tended  to  feel 
that  the  user  had  stated  clear  and  specific  project  require- 
ments where  there  was  a  high  level  of  user  participation,  as 
indicated  by  the  fact  that  most  of  the  variables  shown  in 
Tables  44  and  45  are  the  same. 

On  the  other  hand,  the  users'  evaluation  of  the 
specificity  of  their  original  desires  and  requirements  was 
related  to  a  separate  group  of  variables  as  shown  in  Table  46. 
This  evaluation  was  not  significantly  related  to  the  project 
leader's  view  of  ostensibly  the  same  factor,  specificity  of 
user  requirements.   It  appeared  that  users  felt  they  had  to 
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be  more  specific  in  stating  what  they  wanted  in  larger  organize- 
tions  to  get  their  projects  accepted.   It  also  appeared  that 
the  users  were  more  accurate  than  project  leaders  in  their 
appraisals  of  how  clearly  they  stated  exactly  what  they  wanted 
from  projects.   Although  not  significant  at  the  .10  level, 
the  users'  view  of  specificity  of  their  requirements  (Var.  45) 
was  related  to  the  time  success  criterion  (Var.  48)  with  a  tau 
of  -.272,  a  normal  deviate  of  1.586,  and  a  probability  of  .114. 
Further,  there  was  a  significant  relationship  between  Variable 
45  and  both  the  elapsed  months  spent  on  the  project,  and  the 
project  leader's  appraisal  of  time  success. 

TABLE  44 
Specificity  of  User  Requirements -PL  (Var.  38) 
(Two-tailed  significance  at  or  beyond  .10  only) 

Var.     Tau     Z(s)     P(Z) 

Project  Team  14  .750         3.032  .0027 

User  Man  Months/ 

Total  Man  Months  15  .562         3.073  .0022 

User  Participation-PL  41  .562  3.036  .0025 

Implementation  Problems-User  53  .419  2.290  .021 

User  Sat is  fact ion -PL  57  B.409  2.116  .036 

User  Participation -User  46  .400  2.160  .030 

Project  Success-PL  56  .367  1.937  .052 

User  Management  Interest  39  .320  1.675  .094 

Two  or  More  Years  Systems 

Experience  27    -.450    2.435    .014 
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TABLE  45 
User  Participation -PL   (Var.   41) 
(Two-tailed  significance  at  or  beyond   .10  only) 


Var. 


Tau  Z(s)  P(Z) 


Project  Team 

Specificity  of  User 
Requ  ir  emen  t  s -PL 

User  Man  Months/ 


14 


38 


,660         2.603  .009 


562         3.036  .0025 


Total  Man  Months 

15 

.539 

3.233 

.0014 

User  Management  Interest 

39 

.453 

2.332 

.020 

User  Satisfaction-PL 

57 

.394 

2.135 

.032 

User  Participation -User 

46 

.367 

2.174 

.030 

Measurable  Project  Objectives 

12 

.344 

1.830 

.068 

Two  or  More  Years   Systems 

Experience 

27 

-.361 

2.141 

.032 

TABLE  46 
Specificity  of  User  Requirements -User    (Var.    45) 
(Two-tailed  significance  at  or   beyond    .10  only) 


Var. 


Tau 


Z(s)  P(Z) 


Employees 

3 

B.405 

2.316 

.020 

Time  Success -PL 

49 

.394 

2.114 

.034 

User  Participation -User 

46 

.356 

2.111 

.034 

Elapsed  Months 

21 

-.300 

1.756 

.080 

College  Degree 

5 

-.333 

1.977 

.048 
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On  the  assumption  that  clear,  specific,  and  detailed 
user  requirements  provided  the  project  staff  with  a  more  concrete 
nucleus  around  which  to  build  the  project,  time  success  should 
have  been  greater.   The  data  would  appear  to  support  this 
contention. 

User  participation  (Var.  46)  also  appeared  to  be 
greater  when  users  felt  they  had  been  very  specific  in  stating 
their  requirements.   As  was  mentioned  at  an  earlier  point, 
where  the  users  were  able  to  state  exactly  what  they  wanted  from 
a  project,  they  apparently  felt  they  had  control  of  what  was 
developed,  and,  therefore,  that  they  had  high  participation. 

It  may  strike  the  reader  as  odd  that  there  was  no 
relationship  between  reported  specificity  of  user  requirements 
(Var.  45)  and  user  satisfaction  (Var.  54).   However,  a  close 
investigation  revealed  three  projects,  termed  "evolutionary", 
which  were  low  on  Variable  45,  but  high  on  user  satisfaction 
(Var.  54).   There  were  also  three  projects  which  were  high  on 
Variable  45,  but  very  low  on  Variable  54,  a  result,  apparently, 
of  users  not  getting  what  they  had  specified  as  required. 
These  several  projects,  in  effect,  "cancelled  out"  any  relation- 
ship between  specificity  of  user  requirements  (Var.  45)  and 
user  satisfaction  (Var.  54)  in  the  computation  of  the  tau 
statistic. 

In  summary,  whereas  the  project  leader  appeared  to 
base  his  evaluation  of  the  specificity  of  user  requirements 
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on  the  degree  of  user  participation  in  the  project,  the  users 
seemed  to  base  the  appraisal  of  their  participation,  in  part, 
on  how  specifically  they  stated  their  requirements  in  the  first 
place.   There  were  cases,  however,  where  users  were  not  specific 
in  stating  their  requirements,  but  felt  they  participated 
through  an  evolutionary  process  of  trial  and  error  over  a 
relatively  long  period  of  project  development.   Finally,  users 
and  project  leaders  did  not  tend  to  agree  on  the  degree  of 
specificity  in  original  project  requirements,  but  the  data 
suggest  that  perhaps  the  users'  appraisals  were  more  accurate 
than  those  of  project  leaders.   This  last  point,  on  the  accuracy 
of  appraisals,  should  be  researched  more  thoroughly,  however, 
before  any  firm  conclusion  is  drawn. 

Quality  of  Documentation 

For  the  projects  in  the  study,  the  perceived  quality 
of  documentation  was  not  related  to  the  presence  or  absence  of 
documentation  standards.  Again,  it  is  emphasized  that  only 
four  projects  were  developed  without  documentation  standards, 
so  it  would  seem  unreasonable  to  conclude  that  documentation 
tends  to  be  as  good  without  standards  as  with  them.   However, 
it  did  appear  that  the  mere  presence  of  standards  was  not  enough 
to  assure  high  quality  documentation. 

As  shown  in  Table  47,  the  perceived  quality  of  project 
documentation  wa   elated  to  several  other  variables.   It  has 
already  been  pointed  out  that  both  users  and  project  leaders 
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perceived  fewer  Implementation  problems  where  documentation  was 
of  relatively  high  quality.   This  finding  was  not  unexpected. 

TABLE  47 
Quality  of  Documentation  (Var.  36) 
(Two-tailed  significance  at  or  beyond  .10  only) 


-PL 

Var. 

Tau 

Z(s) 

P(Z) 

Implementation  Problems 

58 

.533 

2.869 

.004 

Implementation  Problems 

-User 

53 

.333 

1.739 

.082 

High  Level  Programming 
Language 

43 

.300 

1.780 

.076 

Man  Months 

22 

-.327 

1.663 

.096 

Project  Control 

42 

B-.336 

1.650 

.10 

Elapsed  Months 

21 

-.340 

1.736 

.082 

The  relationship  of  perceived  documentation  quality 
to  the  use  of  higher  level  programming  languages,  essentially 
COBOL,  has  also  been  discussed.   It  appeared  that  most  project 
leaders  who  used  COBOL  felt  they  had  adequate  documentation  to 
allow  reasonably  easy  program  maintenance,  which  seems  to  have 
been  their  primary  criterion  for  quality. 

Finally,  the  perceived  quality  of  project  documenta- 
tion was  negatively  related  to  the  elapsed  time  on  the  project, 
the  man  months  on  the  project,  and  reported  project  control. 
These  rather  weak  relationships  could  be  interpreted  at  least 
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two  different  ways.  One  interpretation  would  be  that  projects 
took  longer  because  of  poor  quality  documentation  that  occasioned 
delays,  duplication,  and  general  confusion.   Another  interpreta- 
tion, which  seems  to  make  some  sense,  would  be  that  where  project 
control  schemes  were  more  stringent,  pressure  built  up  on  project 
leaders  and  staffs  as  the  project  took  longer  or  more  man 
months  were  expended.   The  response  to  the  pressure  was  to  cut 
corners  on  documentation,  concentrating  on  getting  something 
running  as  quickly  as  possible.   If  the  latter  possibility  were 
true,  the  control  schemes  used  could  have  been  dysfunctional 
to  long  range  project  success.  Most  of  the  prescriptive  litera- 
ture argues  for  both  documentation  standards,  to  assure  high 
quality  documentation,  and  project  control  techniques,  to  keep 
projects  on  target.   They  may  not  be  compatible.   This  is  a 
very  interesting  question  which  should  be  investigated  by 
experimental  means  if  possible. 


PART  IV 

SUMMARY  OF  RESULTS,  CONCLUSIONS,  AND 
SUGGESTED  FUTURE  RESEARCH 
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CHAPTER  VIII 
SUMMARY  OF  RESULTS 

Introduction 

In  Chapter  I  the  purpose  of  the  research  described  In 

this  thesis  was  phrased  as  the  question: 

What  organizational  and  procedural  factors 
are  correlates  of  success  with  MIS  projects? 

In  seeking  answers  to  that  question,  the  literature  on  MIS, 

project  management,  and  Information  systems  In  general  was 

reviewed.  While  few  satisfactory  answers  were  found  In  the 

literature  review,  a  set  of  thirty-five  hypotheses  was 

developed  which,  it  was  hoped,  could  be  tested. 

The  list  of  hypotheses  was  pared  down  from  thirty-five 
to  sixteen  after  analyzing   the  responses  of  information  systems 
professionals    to  a  request  to  rank  the   thirty-five  hypotheses, 
or   factors,    in   terms  of  their   importance  to  MIS  project  success. 
These  sixteen  hypotheses,   along  with  four  criteria  of  MIS  project 
success,    then  became   the  nucleus  of  a  questionnaire  which  was 
developed  for   interviewing  people  in  organizations  who  had  been 
involved  with  MIS  projects. 

A  successful  pre-test  of  the  questionnaire  was   followed 
by  the  selection  of  a  study  sample.      Ten  organizations   from  the 
Twin  Cities,   representing  seven  Industries,  were  chosen  for   the 
study.      Data  were  collected  on  two  MIS  projects   In  each  of   these 
organizations   between  October  and  mid -December,    1970. 
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The  data  collected  were  analyzed  to  determine  what 
relationships  existed  among  variables  in  the  study.   The  primary 
form  of  statistical  analysis  was  the  computation  of  Kendall's 
rank  correlation  statistic,  tau,  for  every  possible  relationship 
between  any  two  of  the  sixty-one  variables  identified  in  the 
study. 

The  results  of  the  data  analysis  have  been  discussed 
in  detail  in  Chapters  VI  and  VII.  This  chapter  is  devoted  to 
summarizing  the  findings  of  the  study.   The  final  chapter, 
Chapter  IX,  will  be  devoted  to  drawing  conclusions  from  the 
results,  and  to  suggesting  some  possible  future  research 
directions  based  on  what  has  been  found  here. 

Independence  of  Criteria  of  Success 
Four  criteria  of  MIS  project  success  were  posited  and 
used  in  this  study:   time  success,  cost  success,  user  satisfac- 
tion, and  computer  operations  success.   These  four  criteria  were 
found  to  be  reasonably  independent  measures  of  MIS  project 
success.   The  closest  relationship  found  between  any  two  criteria 
was  that  between  user  satisfaction  and  computer  operations  success, 
This  would  seem  to  be  a  logical  relationship.  Where  a  project 
impacts  on  computer  operations  in  such  a  way  that  scheduling 
and  running  problems  develop,  the  users  probably  feel  such 
problems  through  inadequate  response  and  service  from  the 
information  systems  function. 
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Flndings:   Tests  of  Hypotheses 
Hypotheses  Related  to  Criteria  of  Success 

Of  the  sixteen  hypotheses  which  were  selected  to  be 
tested  In  the  present  study,  one  was  not  effectively  tested 
by  the  sample  data,  ten  were  found  to  be  significantly  related 
to  at  least  one  criterion  of  success,  and  five  were  found  to 
be  unrelated  to  any  criterion  of  success.   The  hypothesis 
which  was  not  tested,  and  about  which  no  conclusions  could  be 
drawn,  was  the  degree  of  relationship  between  centralization 
of  the  Information  systems  function  and  MIS  project  success. 

The  ten  hypotheses  which  were  tested  and  found  to 
be  related  to  project  success  are  shown  In  Table  48.   For 
each  entry,  the  rank  of  the  hypothesis,  In  terms  of  Its  importance 
to  project  success,  from   the  original  ranking  of  thirty-five 
hypotheses  is  shown  first;  next  is  shown  the  hypothesis  itself; 
and  shown  last  are  the  separate  criteria  to  which  the  hypothesized 
factor  was  related.  A  'V  in  the  table  indicates  that  the 
factor  contributed  to  success  on  the  criterion  in  question, 
and  a  "-"  indicates  that  the  factor  detracted  from  success  on 
the  criterion  in  question.   Tables  16-19  in  Chapter  VI  should 
be  consulted  to  find  the  strengths  of  the  relationships  shown 
in  Table  48,  and  to  determine  the  variable  numbers  used  in  the 
study  to  represent  the  hypotheses.  Further,  the  discussions 
in  Chapter  VI  concerning  Tables  16-19  should  be  referenced  for 
Interpretation  of  the  relationships. 


TABLE  48 
Summary  of  Findings 


-165- 


User 

Satis-    Computer 


Rank      Hypothesis 

Time 

Cost 

faction  Operations 

1  Participation  by  operat- 
ing management  in  design, 
formal  approval  of 
specifications,  and 
continual  review  of 
project. 

+ 

7  Organization  level  of 
top  computer  executive. 

— 

+ 

12  Documentation  standards 
used  and  enforced. 

+ 

14  Low  turnover  of  project 
personnel. 

+ 

16  Source  of  origination  of 
project  (MIS  staff  or 
user). 

+ 

20  Length  of  experience  in 
the  organization  of 
project  personnel. 

— 

+ 

23  High  level  programming 
language  used  for 
project. 

+ 

31  High  formal  educational 
level  of  project 
personnel. 

"'  " 

+ 

32  Separation  of  analysts 
and  programmers  for 
large  projects. 

+ 



33  Overall  size  of  organi- 
zation systems  staff. 



—~~ 

+ 
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Hypo theses  Not  Related  to  Criteria  of  Success 

The  five  hypotheses  found  to  be  unrelated  to  project 
success  for  the  projects  in  the  sample  were: 
Rank  Hypothesis 

2  Measurable  project  objectives  from 
conception  of  the  project. 

3  Utilization  of  a  project  team  composed 
of  MIS  staff  and  user  personnel. 

8      Systems  experience  of  project  personnel. 

13      Use  of  a  formalised  and  regular  reporting 
structure  on  project  progress. 

34      Ratio  of  computer  hardware  investment  to 
total  sales  or  operating  budget. 

General  Comments  Concerning  the  Hypotheses 
In  reviewing  the  above  findings  from  the  study,  one 
impression  that  immediately  comes  to  the  fore  is  the  error  in 
the  evaluations  of  the  information  systems  professionals  who 
ranked  the  thirty-five  hypotheses  in  terms  of  contribution  to 
project  success.  More  of  the  hypotheses  were  confirmed  from 
the  lower  half  of  the  ranking  than  from  the  top  half.   In  other 
words,  those  factors  which  the  information  systems  professionals 
thought  most  crucial  to  MIS  project  success  tended  to  be  rejected 
more  than  those  factors  thought  least  crucial.   However,  the 
questions  and  possible  explanations  arising  from  this  situation 
are  more  interesting  than  the  situation  Itself. 

A  fundamental  benefit  to  the  researcher  in  conducting 
this  study  was  that  the  general  confusion  which  surrounds  the 


-167- 

definltion  of  MIS  was  brought  into  clearer  focus.  That  there 
Is  rather  widespread  disagreement  among  those  in  the  informa- 
tion systems  community  concerning  what  MIS  really  is,  or  ought 
to  be,  became  apparent  in  analyzing  the  responses  of  the 
information  systems  professionals  who  attended  the  Founding 
Conference  of  Society  for  Management  Information  Systems  in 
1969  to  a  request  to  define  what  MIS  meant  to  them.   The 
different  definitions  of  MIS  were  almost  as  numerous  as  the 
respondents  themselves  (Dickson,  1970). 

The  researcher  tried  to  avoid  this  confusion  from 
the  beginning  of  this  study  by  concentrating  on  MIS  projects, 
and  by  defining  rather  explicitly  the  criterion  which  placed 
a  project  in  the  management  information  system  category.  Very 
simply,  any  project  selected  for  study  had  to  result  in  infor- 
mation being  provided  to  a  manager,  at  some  level,  which  was 
used  by  that  manager  in  the  dec is  ion -making  process  relative 
to  his  domain  of  responsibility.   The  emphasis  throughout  the 
study  was  on  looking  at  projects  which  had  been  conceived  to 
furnish  managers  information  that  would  assist  them  in  making 
decisions  in  a  more  effective  way. 

The  conclusion  reached  during  the  course  of  the 
research  was  that  there  are  at  least  three  different  types  of 
computer -oriented  information  system  projects  which  should  not 
be  lumped  together.   These  three  are: 
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1.  Data  processing  projects,  where  the  basic 
objective  is  to  process  operational  data  in 
a  more  efficient  manner  by  using  a  computer 

in  lieu  of  manual  or  accounting  machine  methods. 

2.  Generalized  software  projects,  where  the  object- 
ive is  to  produce  a  marketable  software  package 
for  widespread  consumption  by  various  users. 
There  is  no  explicit  concern  here  for  the  specific 
information  needs  of  one  or  a  few  users  in  one 
organization,  although  the  products  of  such  an 
endeavor  might  facilitate  responding  to  those 
needs  after  further  development,  modification, 

or  incorporation  into  a  user -oriented  system. 

3.  Management  information  system  projects,  where 
the  objective  is  to  provide  a  manager,  or 
managers,  with  specific  information  system 
products  that  will  enable  those  managers  to  do 
a  better  job  of  dec is  ion -making  relative  to 
specific  kinds  of  problems. 

As  has  already  been  pointed  out,  this  research  has 
focused  entirely  on  category  3  above.   However,  the  specification 
of  a  definition  for  the  purposes  of  this  study  can  in  no  way  be 
expected  to  go  beyond  the  bounds  of  this  study.   In  other  words, 
although  a  definition  of  MIS  projects  has  been  provided  which 
has  suited  the  purposes  here  very  well,  it  is  only  one  of  many 
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posstble  definitions.   The  data  and  results  presented  in  this 
thesis  are  relative  to  only  one  portion  of  the  information  systems 
spectrum,  and  where  factors  were  ranked  by  others  in  the  field 
as  to  their  Importance  to  MIS  project  success,  other  frames  of 
reference  were  possible.   Therefore,  to  say  that  the  conclusions 
reached  here  serve  to  reject  the  general  thinking  of  professionals 
in  the  information  systems  field  as  to  what  factors  are  critical 
to  MIS  project  success  would  be  an  error. 

With  that  background,  several  further  comments  can  be 
made  about  the  findings  and  conclusions  presented  here.   Since 
the  entire  focus  of  the  present  study  has  been  MIS  projects, 
as  they  have  been  defined,  all  of  the  findings  must  be  viewed 
in  that  context.   The  size  and  scope  of  projects,  how  they 
were  managed,  the  interactions  that  occurred,  and  measures  of 
success  must  all  be  considered  in  the  context  of  the  MIS  project. 
This  means  that  generalizations  beyond  this  type  of  project 
should  only  be  made  with  the  greatest  caution,  if  at  all.   This 
is  not  to  say  that  the  findings  here  are  of  no  consequence  because 
they  don't  apply  to  the  entire  information  systems  spectrum. 
Rather,  other  portions  of  that  spectrum  must  be  investigated 
separately,  or  at  least  with  an  awareness  that  there  seem  to 
be  fundamental  differences  in  what  one  is  looking  at.   However, 
even  this  last  statement  cannot  be  confirmed  until  these  other 
segments  of  the  information  systems  spectrum  have  been  analyzed 
in  somewhat  the  same  way  that  MIS  projects  have  been  analyzed 
in  this  study. 
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The  User  Participation  Cluster 
It  was  definitely  concluded  from  the  analysis  of 
the  data  collected  in  the  study  that  perceived  user  participa- 
tion is  a  crucial  factor  in  the  success  of  MIS  projects.  There 
was  a  cluster  of  variables  that  seemed  to  represent,  essentially, 
the  direct  and  close  involvement  of  users  in  the  development  of 
their  projects.   This  cluster  consisted  of  origin  of  the 
project,  reported  clarity  of  objectives,  reported  specificity 
of  actual  information  requirements,  the  use  of  an  inter functional 
project  team,  and  finally,  the  user  participation  variable 
itself.   However,  making  this  general  statement  necessitates 
explanation  of  several  points  with  respect  to  success  and  the 
separate  variables  just  cited. 

First,  there  is  an  implicit  value  judgment  that  user 
satisfaction  with  a  MIS  project  is  the  most  important  success 
criterion.   This  is  based  on  the  nature  of  the  MIS  project  as 
opposed  to  other  possible  project  types.   Since  the  primary 
purpose  of  the  MIS  project  is  to  provide  information  to  a 
manager  to  support  his  dec is  ion -making  responsibilities,  the 
failure  of  a  project  to  satisfactorily  serve  this  end  means 
failure  for  the  project  in  terms  of  its  essential  purpose. 
Being  within  budget  on  time  and  cost,  and  creating  no  problems 
for  computer  operations,  are  hardly  meaningful  if  the  completed 
project  is  not  used  or  is  viewed  by  users  to  be  inadequate 
for  their  information  needs.   This  is  not  an  advocacy  of 
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complete  freedom  from  budgets  or  the  need  to  consider  how  a 
given  project  will  affect  computer  operations.   It  Is  merely 
a  value  hierarchy  of  the  criteria,  with  user  satisfaction  as 
the  primary  criterion  and  the  other  three  as  Important,  but 
secondary  criteria  of  success. 

Moving  to  a  closer  consideration  of  the  variables 
Included  In  the  cluster  mentioned  above,  several  Important 
aspects  of  perceived  user  participation  were  brought  out  In 
the  analysis  of  sample  data.   First,  although  clear  and  measur- 
able project  objectives  would  logically  seem  to  be  a  big 
factor  In  users  getting  what  they  could  use,  and  within  time 
and  cost  expenditures  reasonably  close  to  budgets,  this  was 
not  always  the  case.   In  fact,  there  were  enough  projects  in 
the  study  which  did  not  follow  this  logical  pattern  that  the 
nature  of  project  objectives  was  not  significantly  related  to 
any  criterion  of  success.   Those  projects  which  did  not  fit 
the  expected  pattern  were  of  two  types:   projects  of  the  "evolu« 
tionary"  type,  and  projects  where  users  knew  what  they  wanted 
to  be  able  to  do,  but  did  not  understand  their  own  information- 
decision  environments  well  enough  to  know  how  to  do  it. 

In  the  first  type,  the  "evolutionary"  project, 
objectives  were  often  reported  to  have  been  rather  vague  at 
the  beginning  of  a  project,  but  the  users  worked  so  closely 
with  the  project  staff  that,  over  a  fairly  long  period  of  time, 
during  which  the  users  progressively  learned  more  about  what 
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information  they  could  use  and  In  what  ways,  results  were 
generated  with  which  the  users  were  highly  pleased.   In  the 
second  type,  where  It  was  reported  that  there  were  clear  and 
measurable  project  objectives,  but  where  the  users  were  not 
able  to  specify  precisely  what  Information  they  needed,  or 
In  what  form,  to  reach  those  objectives,  a  good  deal  of 
frustration  and  slippage  occurred  during  project  development. 
It  would  seem  that  In  these  cases  a  project  was  begun  before 
enough  spadework  had  been  done  by  users.   In  an  effort  to 
turn  something  out  the  project  staff,  after  floundering  for 
a  length  of  time,  froze  requirements  which  may  or  may  not  have 
been  what  the  users  ended  up  feeling  they  should  have  gotten 
from  the  project. 

With  respect  to  the  source  or  originator  of  a  project, 
those  initiated  by  users  were  generally  more  successful.   The 
users  usually  knew  what  they  wanted,  and  were  thus  able  to  work 
with  project  staffs  in  getting  it.   However,  where  users  initiated 
a  project,  but  then  did  not  participate  very  much  in  the  develop- 
ment efforts,  results  were  poor.  While  the  lack  of  user  partici- 
pation may  have  resulted,  in  part,  because  the  project  leader 
did  not  allow  it,  the  major  problem  appeared  to  be  users  who 
were  unwilling  to  contribute  in  a  meaningful  way.   As  a  conse- 
quence, the  project  leader  made  decisions  by  default  which 
affected  the  acceptability  of  the  final  products  in  the  eyes  of 
the  users. 
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From  the  preceding  comments  one  draws  the  conclusion 
that  some  organizations,  and  some  individuals  in  organizations, 
have  not  come  to  recognize  a  difference  between  MIS  projects  and 
data  processing  projects.   This  situation  was  nowhere  more 
apparent  than  in  the  findings  regarding  the  use  of  inter functional 
project  teams.  While  teams  did  appear  to  enhance  user  partici- 
pation, there  were  cases  where  the  team  approach  was  implemented 
in  such  a  way  that  project  success  was  marginal,  if  not  poor. 
There  were  enough  of  these  cases  that  project  success  and  the 
use  of  teams  were  not  significantly  related  to  each  other. 

It  was  found  that  merely  setting  up  a  project  team 
with  users  represented  does  not  insure  participation  of  the 
ultimate  users  of  project  outputs.   In  several  cases,  the  data 
processing  orientation  seemed  to  dominate.   It  was  assumed  by 
user  management,  erroneously,  that  merely  having  someone  on  the 
project  staff  from  the  user  area  meant  there  would  be  high  parti- 
cipation, and  the  user  management  would  end  up  with  satisfactory 
results.   This  approach  may  have  served  quite  well  for  data 
processing  projects,  where  the  primary  focus  was  on  getting 
procedural  aspects  of  an  operational  function  cut  over  to  the 
computer.   A  staff  specialist  from  the  user  area  who  knew  the 
procedures,  records,  and  so  forth,  could  do  an  excellent  job 
for  the  user  function  in  working  as  a  member  of  the  data 
processing  project  staff.   However,  where  the  project  is 
primarily  oriented  to  providing  a  manager  with  information  to 
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asslst  his  decision  making,  that  manager,  not  the  staff 
specialist,  should  be  the  one  involved.   The  fact  that  in 
some  cases  this  did  not  occur  gave  rise  to  the  disagreement 
found  among  user  personnel  interviewed  about  the  same  project. 
The  staff  person  from  the  user  area  felt  he  had  participated 
heavily,  and  the  results  of  the  project  were  very  good,  whereas 
a  manager,  who  was  actually  receiving  the  project  outputs,  felt 
participation  was  not  so  high  and  the  results  were  not  so  good. 

Another  problem,  related  to  the  one  just  discussed, 
was  the  case  of  the  projects  where  the  user  representatives  who 
worked  on  the  project  staff  were  not  the  people  who  actually 
implemented  the  project.  For  one  reason  or  another,  the  user 
representative  was  given  other  tasks,  not  directly  related  to 
the  MIS  project  studied,  soon  after  the  project  had  been 
completed.   This  meant  that  people  unfamiliar  with  what  had 
transpired  during  project  development  were  forced  to  implement 
the  project,  and  to  maintain  continuing  liaison  with  the 
information  systems  function.   The  results,  not  unexpectedly, 
were  rather  poor.   The  situation  just  described  occurred  either 
because  the  user  representatives  on  the  project  team  were  staff 
personnel  who  were  put  on  something  else  as  soon  as  the  project 
was  completed,  or  the  managers  who  had  wanted  the  project 
developed,  and  worked  with  the  project  team,  were  transferred 
shortly  before  or  after  project  implementation. 
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The  Project  Management  Cluster 

Three  variables  were  viewed  as  comprising  a  project 
management  cluster:   documentation,  project  control,  and  the 
choice  between  separate  or  combination  analyst/programmers. 
All  three  of  these  factors  have  been  subjects  of  considerable 
discussion  In  the  Information  systems  literature,  as  pointed 
out  In  Chapter  11.   It  would  appear  that  much  of  the  clamor 
surrounding  these  three  factors  has  arisen  as  a  result  of 
the  difficulties  experienced  with  the  development  of  data 
processing  projects  and  generalized  software  projects.   However, 
no  distinction  has  generally  been  made  among  the  different 
kinds  of  project  environments  In  prescribing  what  those 
Involved  In  developing  computer  based  systems  should  do.   The 
Implicit  assumption  seems  to  have  been  that  all  projects  are 
essentially  the  same,  and,  therefore,  the  same  nostrums  which 
were  believed  to  be  useful  In  remedying  data  processing  and 
generalized  software  development  ills  would  be  applicable  In 
the  management  Information  system  environment. 

There  was  certainly  no  evidence  in  this  study  which 
leads  one  to  advocate  anarchy  in  the  management  of  MIS  projects; 
which  nudges  one  toward  the  conclusion  that  project  management 
efforts  are  futile  and  should  be  abandoned.   However,  there 
was  evidence  to  support  the  position  that  MIS  project  manage- 
ment should  be  considered  in  the  MIS  context.   Unfortunately, 
in  this  regard,  about  the  best  the  data  analysis  from  this 
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study  can  do  Is  raise  some  questions  which  need  to  be  investi- 
gated thoroughly. 

What  are  these  questions  that  analysis  of  the  data 
collected  have  brought  to  focus?  First,  is  the  issue  of 
assigning  separate  analysts  and  programmers  to  a  MIS  project 
versus  assigning  one  person  who  performs  both  functions.   This, 
of  course,  does  not  mean  that  only  one  person  would  be  assigned 
to  a  project  in  all  cases;  rather,  that  the  project  would  be 
divided  into  functional  modules  where  one  individual  takes 
responsibility  for  all  analysis  and  programming  involved  for 
assigned  modules. 

Over  the  past  several  years,  the  sentiment  has 
shifted  from  time  to  time  as  to  which  approach  should  be  used; 
at  no  time  has  there  been  general  agreement  on  which  approach 
is  best.   There  have  been  advocates  in  both  camps.   It  would 
seem  that  this  issue,  as  with  the  others  raised  in  this  chapter, 
is  one  without  a  universal  solution.   What  is  "best"  probably 
depends  on  the  organization  and  information  system  environment. 

At  least  for  the  MIS  environments  of  organizations 
sampled  in  connection  with  the  research  reported  here,  the  use 
of  combination  analyst /programmers  tended  to  yield  the  best 
results  in  terms  of  user  satisfaction  with  project  products. 
This  seemed  to  stem  mainly  from  the  ability  of  the  combination 
analyst/programmer  to  respond  more  quickly  and  effectively  to 
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user  requirements  and  problems.  Where  the  user  was  able  to 
deal  with  one  person  who  had  the  whole  picture  with  respect 
to  one  or  more  functional  modules  of  a  project,  the  user 
appeared  to  be  more  confident  of  what  he  could  expect  from 
the  information  systems  staff  and  tended  to  be  less 
frustrated  by  the  implementation  problems  that  arose. 

To  say  that  all  MIS  projects  should  be  developed 
by  combination  analyst/programmers  would,  of  course,  be  an 
unreasonable  conclusion  to  reach.   Several  organizations  had, 
in  effect,  a  dual  system,  where  some  combination  analyst/ 
programmers  were  used,  but  where  newer  personnel,  trainees, 
were  also  used  as  programmers  only.  This  approach  seems  to 
have  some  merit,  in  that  new  people  must  be  trained  by  some 
means.  The  question  that  should  be  raised,  however,  is  whether 
or  not  this  is  the  most  effective  means  of  training  new  systems 
people.  This  issue  should  be  investigated,  because  it  is 
possible  that  user  satisfaction  with  MIS  project  results  may 
be  adversely  affected  where  the  separate  analyst /programmer 
approach  is  utilized  as  a  means  of  training  new  people.  The 
important  point  is  for  the  management  of  the  information  systems 
function  to  be  aware  of  this  possibility,  so  that  safeguards 
can  be  built  into  project  management  to  minimize  adverse  Impact 
on  user  satisfaction  with  project  results. 

A  second  aspect  of  MIS  project  management  which  would 
seem  to  require  a  reappraisal  is  project  control  techniques. 
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For  the  projects  in  the  sample,  the  project  control  variable 
was  not  related  to  any  criterion  of  success,  and  it  was  negatively 
related  to  the  project  leader's  appraisal  of  documentation  quality. 
These  findings  could  be  interpreted  in  several  ways.   However,  an 
explanation  that  encompasses  all  of  these  findings  on  project 
control  is  simply  that  the  methods  of  project  control  were  not 
conceived  nor  applied  in  the  MIS  context. 

Where  stringent  progress  reporting  against  planned 
activities  was  required,  that  reporting  did  not,  in  general, 
result  in  actual  control  being  exercised.   In  the  case  of  only 
one  project  were  additional  resources  committed  to  enable  the 
project  staff  to  achieve  the  target  date  for  completion.   In  the 
other  cases,  progress  information  seems  to  have  been  collected 
for  its  own  sake,  with  little  apparent  use  made  of  the  information 
for  control  purposes.  Where  target  dates  were  revised  backwards 
as  a  result  of  slippage,  this  was  of  value  to  those  in  the  organi- 
zation who  were  to  be  affected  by  the  completed  project,  but 
such  use  of  supposed  control  information  would  not  seem  to 
constitute  the  whole  realm  of  project  control. 

On  the  negative  side,  where  stringent  project  status  re- 
porting was  required,  project  staffs  seemed  to  have  felt  pressure 
to  get  programs  running  as  quickly  as  possible.   However,  where 
they  had  no  additional  resources  available  to  them,  about  all 
project  staffs  could  do  to  ease  the  pressure  was  cut  corners 
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on  documentation  and  implementation  preparations,  and  refuse 
to  make  changes  requested  by  users.   In  other  words,  it 
appeared  that  project  staffs  were  frustrated  by  control  methods 
rather  than  helped  by  them. 

The  reader  may  feel  that  the  application  of  pressure 
as  a  means  of  reaching  target  dates  is  useful,  nonetheless; 
that  without  such  reporting  and  pressure,  projects  could  go 
on  for  longer  periods  than  they  already  do.   And  it  is 
certainly  true  that  with  respect  to  the  projects  in  this 
sample  which  were  subject  to  tight  progress  reporting  require- 
ments, the  actual  times  may  have  been  less  than  they  would 
have  been  had  there  existed  no  progress  reporting.   It  is 
impossible  to  resolve  that  question,  of  course.   The  really 
important  point  in  all  of  this,  however,  is  the  relation  of 
the  pressure  to  the  quality  of  the  original  project  estimate. 

It  appeared  to  the  researcher  that  project  time  and 
cost  estimates  were  probably  made  relative  to  data  processing 
projects  rather  than  to  MIS  projects.   In  very  few  cases  was 
there  recognition  at  the  beginning  of  a  project  that,  in  the 
MIS  environment,  managers  must  learn  to  use  what  they  receive; 
that  they  may  want  to  change  the  content  or  format  of  outputs 
as  they  go  along  to  whatever  is  easiest  for  them  to  use  or 
fits  into  their  dec is  ion -making  styles.   Therefore,  as  a  result 
of  poor  estimates  in  the  first  place,  some  project  staffs  were 
subject  to  pressure  to  complete  projects  within  artificial  time 
frames . 
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Finally,  with  respect  to  perceived  quality  of  project 
documentation,  and  documentation  standards,  no  significant  rela- 
tionship was  found  between  these  two  variables.   Although  807.  of 
the  projects  in  the  sample  were  subject  to  some  form  of  documen- 
tation standards,  the  perceived  quality  of  documentation  was  not 
assured  by  such  standards.   It  should  be  pointed  out  that  a  weak- 
ness of  the  present  study  was  the  absence  of  any  objective  measure 
of  project  documentation  which  would  allow  the  comparison  of  all 
projects  against  the  same  specifications.   However,  the  evaluation 
of  project  documentation  by  project  leaders  was  valid  to  the  extent 
that  projects  free  of  documentation  problems  did  seem  to  meet  a 
functional  test  of  documentation  effectiveness.  The  assumption 
was  made  that  where  there  were  no  documentation  problems  perceived 
by  those  who  were  using  the  documentation,  the  documentation  met 
the  functional  test  of  quality. 

The  above  conclusions  are  not  intended  as  a  case  for 
no  standards  or  no  control  with  MIS  projects.  Rather,  the 
point  to  be  made  is  that  both  control  and  documentation  standards 
should  be  conceived  and  applied  to  MIS  projects  with  an  explicit 
recognition  that  the  MIS  project  is  different  from  the  data 
processing  project  or  the  generalized  software  project. 
Documentation  standards  should  be  oriented  to  providing  clear 
understanding  among  all  those  concerned  with  a  project,  rather 
than  to  fixing  responsibility  for  who  said  what  to  whom.  The 
whole  thrust  of  the  standards  should  be  toward  making  it  easier 
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for  evolution  with  users  to  occur  rather  than  Inhibiting  it. 
The  standards  should  be  such  that  change  is  facilitated,  rather 
than  prevented  through  freezing  requirements  too  rigidly  at 
some  arbitrary  point.   Project  documentation  is  certainly  no 
less  important  because  the  project  is  oriented  to  providing 
information  to  managers  to  assist  their  dec is ion -making. 
This  was  borne  out  by  the  relationships  between  the  perceived 
quality  of  project  documentation  and  both  users'  and  project 
leaders'  views  of  implementation  problems.   However,  the 
kinds  of  documentation  standards  which  have  been  applied  do 
not  appear  to  be  the  most  effective  for  assuring  the  type 
of  documentation  needed  with  MIS  projects. 

The  same  sort  of  argument  holds  for  project  control 
methods.   All  control  should  not  go  out  the  window  because  the 
MIS  project  is  somehow  different  from  the  kinds  of  projects 
control  schemes  have  been  devised  to  handle.   In  fact,  it  is 
quite  possible  that  MIS  project  control  will  provide  a 
greater  challenge  to  information  systems  managers  than  data 
processing  projects  have.  Where  time  estimates  cannot  be 
fixed  as  easily,  and  where  several  projects  are  running  at 
one  time  in  various  stages,  the  allocation  and  control  of 
resources  will  likely  be  more  difficult.   But  the  point  is, 
that  if  managers  are  to  derive  truly  usable  products  from 
MIS  projects,  they  should  be  allowed  to  grow  and  experiment 
with  results.   This  means  a  more  fluid  situation  where  estimates 
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are  not  based  on  just  the  apparent  facets  of  a  project  at 
original  definition.   The  estimates  will  also  have  to  be  based 
on  the  kinds  of  products  involved  and  the  individual  situations 
of  the  managers  who  are  to  be  served.   The  notion  of  many  open- 
ended  projects,  current  over  a  long  period  of  time,  may  seem 
to  be  an  impossible  situation  with  which  to  live.   But  what 
is  the  point  of  imposing  artificial  constraints  from  another 
environment  on  MIS  projects,  only  to  achieve  time  and  cost 
'/success"  for  results  that  are  ineffective  or  not  used? 

The  Project  Leader's  View  of  Project  Success 

It  has  been  argued  here  that  the  most  crucial  criter- 
ion of  MIS  project  success  is  user  satisfaction  with  the 
results  of  the  project.   How  do  project  leaders  view  project 
success?   It  would  appear,  based  on  the  sample  data,  that 
project  leaders  implicitly,  if  not  explicitly,  understood  the 
ascendency  of  user  satisfaction  over  other  criteria  for  MIS 
projects.   Project  leaders'  perceptions  of  project  success 
were  related  very  strongly  to  how  satisfied  users  were  with 
project  results.   Further,  project  leaders  clearly  recognized 
the  great  importance  users  attached  to  implementation  problems. 
Where  users  were  able  to  use  project  outputs  with  relative 
ease,  project  leaders  felt  the  projects  were  successful.  Most 
project  leaders  also  seemed  to  recognize  that  cluster  of  user 
participation  variables,  so  important  to  user  satisfaction, 
as  important  in  contributing  to  project  success  from  their  own 
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viewpoints. 

While  project  leaders  did  accord  time  success  some 
Importance  in  evaluating  overall  project  success,  this  was 
secondary  to  user  satisfaction.   Further,  the  project  leader's 
view  of  project  success  was  not  significantly  related  to  either 
cost  success  or  computer  operations  success.   In  short,  project 
leaders  were  keenly  aware,  in  most  cases,  of  how  the  users  felt 
about  projects,  and  these  users  attitudes  were  the  most  impor- 
tant key  in  how  successful  the  project  leaders  felt  the  projects 
were. 

Differing  Views  of  Users  and  Project  Leaders 

Although  users  and  project  leaders  agreed  very  well 
on  what  made  MIS  projects   successful,  they  disagreed  on  two 
important  aspects  of  MIS  projects.  Project  leaders  appeared 
to  consider  implementation  problems  as  those  difficulties 
encountered  in  getting  a  project  completed  and  operational. 
They  tended  to  focus  on  cutover  as  the  crucial  point,  and 
where  problems  arose  in  making  the  cutover,  they  felt  there 
were  implementation  problems.   Users,  on  the  other  hand,  seemed 
to  focus  on  problems  in  using  the  project  products  as  implemen- 
tation problems.   They  tended  to  look  at  implementation  as 
what  occurred  after  cutover;  those  problems  experienced  in  try- 
ing to  work  with  what  had  been  developed  to  assist  them  in 
their  dec is  ion -making  roles. 
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Thls  finding  may  appear  to  be  a  contradiction  of 
what  was  said  earlier  concerning  the  project  leader's  percep- 
tions of  success,  and  how  those  perceptions  were  strongly 
related  to  users  reporting  few  implementation  problems.   However, 
there  really  is  no  contradiction.   The  project  leader,  nurtured 
in  the  data  processing  environment,  tended  to  look  upon  imple- 
mentation as  getting  a  set  of  programs  up  and  running.   Once 
this  was  accomplished,  and  programs  were  reasonably  free  of 
obvious  bugs,  he  and  his  staff  went  on  to  something  else, 
new  projects.   Users,  on  the  other  hand,  had  to  live  with 
what  had  been  given  them.   No  matter  how  free  of  technical 
bugs  a  program  was,  users  had  to  try  to  use  what  they  received. 
Where  there  was  no  follow- through  by  the  systems  staff  In 
helping  users  to  do  this,  users  became  frustrated  and  felt 
there  were  implementation  problems.   Although  project  leaders 
seemed  to  be  aware  of  the  frustrations  and  problems  felt  by 
users,  and  recognized  these  frustrations  and  problems  as 
important  factors  in  how  well  satisfied  the  users  were,  the 
project  staffs  were  often  already  committed  to  other  projects. 
This  meant  that  although  they  recognized  that  follow- through, 
and  perhaps  changes,  were  necessary  to  help  users  get  the  kinds 
of  information  needed,  they  were  not  free  to  do  anything  about  it. 

A  second  area  where  users  and  project  leaders  disagreed 
to  some  extent  was  in  evaluating  how  specific  original  user 
requirements  had  been.   The  project  leaders  tended  to  lump 
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specificity  of  user  requirements  in  with  the  cluster  of  user 
participation  variables,  whereas  users  appeared  to  evaluate, 
in  retrospect,  how  clearly  they  were  able  to  state  what  they 
wanted  at  the  beginning  of  the  project. 

The  evaluations  by  users  of  how  specifically  they 
had  stated  what  they  wanted  were  probably  colored  by  the 
growth  process  in  working  with  outputs.   It  appeared  that 
users  tended  to  be  less  satisfied  after  a  period  of  working 
with  a  project  than  when  the  project  was  just  completed.   This 
was  interpreted  as  a  reflection  of  user  learning.   Thus,  when 
the  user  was  interviewed,  his  feeling  that  he  was  not  presently 
getting  what  he  would  really  like  to  have  probably  influenced 
his  judgment  of  how  clearly  he  had  stated  what  he  wanted  in  the 
first  place.   He  seemed  to  assume  that  if  his  requirements 
were  not  now  being  satisfied  as  well  as  he  would  like,  he  had 
done  a  poor  job  of  saying  what  he  wanted.   This,  of  course, 
was  not  necessarily  the  case,  because  the  user  may  have  stated 
fairly  well  what  he  wanted  when  he  started.   But  through  the 
experience  of  using  project  outputs  to  do  his  job,  his  require- 
ments had  shifted  to  some  degree. 

The  project  leader,  from  his  perspective,  was 
concerned  with  trying  to  pin  requirements  down  so  something 
could  get  programmed.   The  fact  that  he  had  users  working  with 
him  on  a  team,  or  readily  available  to  work  with  his  staff  if 
not  on  a  team,  probably  facilitated  his  getting  specific 
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requirements  defined.   The  fact  that  the  project  leader  sought 
to  pin  down  requirements  was  natural;  It  was  his  job  to  do  so. 
However,  the  error  was  In  assuming  those  requirements  would  not 
shift  after  the  project  was  Implemented.   It  should  not  be 
construed  that  this  was  necessarily  the  project  leader's  error. 
The  very  nature  of  most  data  processing  environments  precluded 
his  going  very  far  beyond  Implementation.   The  evolutionary 
approach  was  not  recognized  nor  built  Into   the  development 
process. 
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CHAPTER  IX 
GENERAL  CONCLUSIONS  AND  SUGGESTIONS 
FOR  FUTURE  RESEARCH 

General  Conclusions 

The  most  fundamental  conclusion  drawn  as  a  result 
of  conducting  the  research  reported  here  is  that  different 
environments  exist  for  different  kinds  of  information  system 
projects.  While  this  may  sound  like  a  truism,  and  a  very 
superficial  result  of  an  extensive  research  effort,  the  impli- 
cations of  that  conclusion  pervade  virtually  all  of  the  findings. 
If  professionals  in  the  information  systems  field  know  that 
different  kinds  of  projects  do  exist,  and  that  they  should  be 
treated  in  different  ways,  that  knowledge  is  not  readily 
apparent  in  either  the  literature  of  information  systems  or 
the  practice  of  those  whose  vocation  is  information  systems. 

It  was  stated  in  Chapter  VIII  that  at  least  three 
different  types  of  Information  system  projects  appear  to  exist. 
These  three  types  of  projects  would  all  seem  to  place  different 
kinds  of  requirements  on  those  responsible  for  managing  their 
development.   The  present  study  was  devoted  solely  to  MIS 
projects,  so  conclusions  should  not  be  generalized  to  other 
types. 

With  MIS  projects,  the  primary  objective  is  to  provide 
Information  to  managers  which  they  can  use  effectively  in 
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carrying  out  their  dec Is Ion -making  responsibilities.   For  this 
reason,  the  satisfaction  of  the  manager  with  the  results  of 
the  MIS  project  should  be  viewed  as  the  most  Important  criterion 
of  project  success.   Time  success,  cost  success,  and  computer 
operations  success,  while  important,  are  secondary  to  user 
satisfaction. 

To  achieve  success  with  MIS  projects,  the  findings 
in  this  study  suggest  the  following  approaches  to  managing 
project  development: 

1.  Adopt  the  evolutionary  concept  of  project 
development.   This  implies  recognition  of 
a  learning  process  among  users  which  may 
only  begin,  but  is  certainly  continued, 
after  the  project  has  been  initially  cut 
over  and  the  user  is  receiving  outputs. 
Where  the  main  objective  of  a  project  is 
to  enable  a  dec is  ion -maker  to  function 
more  effectively,  the  dec is ion -maker  must 
be  given  the  greatest  precedence  in 
project  development  plans.   Projects  should 
be  considered  from  their  conception  as  rather 
fluid,  with  provision  for  a  trial  and  error 
process  of  growth  on  the  part  of  the  user. 
The  project  should  not  be  viewed,  in  most 
cases,  as  a  one -shot  development  effort 
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whlch  has  a  specific  end  point. 

2.  Because  of  the  nature  of  the  MIS  project, 
follow -through  by  the  Information  systems 
staff  Is  Imperative.   Close  Involvement 
with  the  user  should  not  terminate  upon 
Initial  cutover,  but  should  continue  to 
be  supported  by  systems  staff  resources 
over  a  fairly  long  period  of  time.   It 
would  be  expected  that  this  resource 
commitment  would  decline  as  the  user 
gains  experience  and  confidence  with  the 
project  outputs.   Changes  to  project 
outputs  would  tend  to  be  greater  at  first, 
decreasing  in  scope  as  the  manager  sights- 
ln  on  what  is  really  the  most  valuable 
Information  to  him. 

3.  Project  management  should  be  oriented  to 
the  requirements  of  MIS  projects,  as 
opposed  to  attempting  to  manage  all  informa- 
tion systems  projects  the  same  way  on  the 
implicit  assumption  all  types  are  essentially 
alike.   Documentation  standards  and  project 
control  techniques  should  be  devised  which 
allow  evolution  rather  than  thwart  It.  Time 
and  cost  estimates  should  be  made  in  light  of 
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the  evolutionary  concept,  and  should  not 
be  used  as  cudgels  to  speed  program  conver- 
sion to  free  resources  for  other  purposes. 

4.  Wherever  possible,  combination  analyst/ 
programmers  should  be  utilized  to  facilitate 
user  satisfaction.   The  ability  of  the  user 

to  look  to  one  person  who  can  respond  knowledg- 
ably  and  quickly  to  his  problems  is  impor- 
tant, and  it  would  appear  that  the  combination 
analyst/programmer  is  better  able  to  do  this 
than  individuals  who  have  done  only  analysis 
or  programming  work  on  the  project. 

5.  Large  projects  which  cover  several  functional 
areas  should  be  avoided.  Such  projects  would 
seem  to  be  too  inflexible  for  MIS,  and  too 

far  removed  from  individual  users.   Blumenthal's 
(1969)  framework  for  MIS  projects  would  seem 
to  be  an  appropriate  model  to  follow;  large 
systems  could  be  broken  into  small  modules, 
which  are  both  more  manageable,  and  more 
meaningful  for  users.   The  large  underlying 
projects  which  are  needed  to  support  MIS 
projects  could  be  approached  as  either  data 
processing  projects  or  generalized  software 
projects.   An  example  of  the  latter  would  be 
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a  generalized  file  management  system  which 
can  support  many  different  MIS  projects. 
6.  User  participation  means  just  that;  those 
who  are  participants  in  MIS  project  develop- 
ment should  be  the  managers  who  will  use  the 
products.   The  technique  of  assigning  a 
user  area  staff  analyst  to  the  project  team 
was  fine  for  data  processing  projects  where 
the  primary  requirement  was  for  user  area 
procedural  knowledge  and  expertise.   However, 
the  MIS  project  focuses  on  the  manager  and 
his  information  needs.   The  definition  of 
those  needs  should  not  be  left  to  someone 
else  if  the  manager  is  to  be  satisfied  with 
what  he  gets. 

The  author  has  no  delusions  concerning  the  difficulty 
of  following  the  above  suggestions.   No  specific  steps  have  been 
outlined  which  make  it  easy  to  implement  any  of  these  things. 
And,  there  is  no  guarantee  that  the  separate  suggestions,  if 
followed,  will  bring  success.   However,  the  Important  findings 
of  this  exploratory  study  are  that  organizations  should  move 
towards  different  sets  of  ground  rules  for  MIS  projects  than 
have  been  prescribed  for  data  processing  projects  or  generalized 
software  projects.   The  objective  of  user  satisfaction  should 
guide  the  thinking  of  information  systems  managers  as  they  seek 
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more  effective  ways  to  use  the  computer  in  their  organizations. 

Suggestions  for  Future  Research 
It  is  believed  that  this  study  has  pointed  up  several 
areas  requiring  more  explicit  and  intensive  research;  areas 
which,  if  researched,  should  provide  some  answers  to  the  thorny 
problems  Inherent  in  pursuing  the  course  that  has  been  outlined 
above.   Among  the  Issues  raised  here  which  should  be  investigated 
thoroughly,  and  by  experimental  means,  if  possible,  are: 

1.  The  kinds  of  Information  used  by  managers  in 
making  decisions  concerning  their  domains  of 
responsibility,  and  the  places  they  get  such 
Information  now. 

2.  The  effects  of  various  project  control  schemes 
on  project  staffs;  the  tradeoffs  among  maintain- 
ing order,  dysfunctional  pressure,  and  willingness 
to  adapt  to  user  desires  which  may  put  the  project 
in  conflict  with  the  control  scheme. 

3.  Related  to  2  above  would  be  research  on  the 
reward  structures  in  various  organizations 
as  they  affect  systems  personnel  engaged  in 
developing  MIS  projects.  What  kinds  of  organi- 
zational rewards  are  tied  to  explicit  and 
implicit  organizational  objectives? 

4.  Techniques  for  allocation  and  management  of 
information  systems  staff  resources  where 
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projects  do  not  "end"  at  an  initial,  clear 
cutover  point.   This  would  encompass  techniques 
for  estimating  project  time  in  the  evolutionary 
approach . 

5.  Techniques  for  integrating  managers  into  the 
development  efforts  on  projects  intended  to 
provide  them  with  information  for  decision- 
making.  What  different  approaches  are  possible, 
and  most  effective,  in  getting  managers  to 
accept  the  kind  of  communication  required  in 
following  the  evolutionary  concept  of  MIS 
project  development? 

6.  If  combination  analyst /programmers  should  be 
used  to  develop  MIS  projects,  how  might  new 
systems  personnel  best  be  trained  to  minimize 
deleterious  effects  on  user  satisfaction? 

7.  What  are  correlates  of  project  success  for  other 
kinds  of  information  system  projects,  such  as 
data  processing  projects  and  generalized  soft- 
ware projects?   In  these  other  project  environ- 
ments, what  should  be  the  criteria  of  success, 
and  how  should  those  criteria  be  viewed  in  terms 
of  importance? 

8.  What  kinds  of  project  documentation  are  most 
crucial  to  MIS  project  success?   Is  it  possible 
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to  devise  measures  of  documentation 

quality  which  can  be  applied  objectively 

to  all  MIS  projects  studied  to  allow 

comparisons  across  projects  and  organizations? 
9.  Is  there  a  means  for  evaluating  MIS  project 

complexity  on  an  objective  basis  allowing 

comparison  across  project  and  organization 

boundaries? 

Most  of  the  above  nine  areas  of  suggested  research  are 
broad.   However,  If  the  present  research  has  brought  these  issues 
into  clearer  focus  so  that  further  research  can  be  conducted 
which  will  provide  meaningful  insights  into  the  management  of 
management  information  systems,  an  important  objective  has  been 
achieved. 
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APPENDIX  A 
SMIS  QUESTIONNAIRE 

One  of  the  most  Important  areas  in  the  MIS  field  is 
that  of  the  management  of  MIS  projects.   Below  are  listed  a 
number  of  factors  which  can  Influence  the  overall  success  of 
MIS  projects.   We  would  like  to  have  your  opinion  concerning 
how  these  factors  influence  MIS  project  success. 

We  recognize  that  "project  success"  is  a  vague  term. 
For  this  reason  we  prefer  that  you  use  your  own  definition  and 
frame  of  reference  for  making  responses.   Likewise,  "project" 
is  vague;  so  assume  that  it  refers  to  a  development  effort  on 
the  part  of  EDP  and/or  user  personnel  requiring  at  least  six 
man-months  to  complete. 
Directions: 

Please  evaluate  each  of  the  thirty-four  listed  factors 
in  terms  of  the  factor: 

"Performance  standards  employed  for  analysts 
and  programmers" 
In  other  words,  is  any  given  factor  cited  below  -  more,  less, 
or  of  equal  importance  -  in  determining  MIS  project  success 
than  is  "performance  standards  for  analysts  and  programmers." 
Please  check  the  appropriate  blank  for  each  factor. 
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Standard  factor:   "Performance  standards  employed 
for  analysts  and  programmers" 


Importance  compared  to  standard: 

much    somewhat    about   somewhat   much 
less      less       same     more     more 


1)  High  formal  educational  level 
of  project  personnel 

2)  Proficiency  of  project  person- 
nel as  judged  by  superiors.  .  . 

3)  Length  of  experience  in  the 
organization  of  project 
personnel  

4)  Systems  experience  of  project 
personnel  

5)  Coordinating  ability  of  project 
leader  (superior's  evaluation). 

6)  Persuasiveness  of  project 
leader  (superior's  evaluation). 

7)  Low  turnover  rate  of  MIS  Dept. 
Staff 

8)  Low  turnover  of  project 
personnel  

9)  High  average  income  level  of 
MIS  Dept.  Staff  

10)  High  rates  of  MIS  Dept.  Staff 
drawn  from  within  the  organi- 
zation  

11)  High  availability  of  computer 
time  for  program  testing.  .  .  . 

12)  Program  maintenance  and  review 
responsibility  specified  for 
definite  period  after  imple- 
mentation ........... 
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Standard  factor:   "Performance  standards  employed 
for  analysts  and  programmers" 


Importance  compared  to  standard: 

much    somewhat    about   somewhat   much 
less      less       same     more     more 


13)  Measurable  project  objectives 
from  conception  of  the  project 

14)  Formal  project  selection 
process  used  to  determine 
which  projects  to  develop.  .  . 

15)  Separation  of  analysts  and 
programmers  for  large  projects 

16)  Combination  analyst/programmer 
for  small  projects  

17)  High  level  programming  lang- 
uage used  for  project 

18)  Formal  training  program  set 
up  for  user  organization  .  .  . 

19)  Number  of  years  experience 

for  organization  with  computer- 
ized information  systems  .  .  . 

20)  Documentation  standards  used 
and  enforced  

21)  Utilization  of  a  project  team 
composed  of  MIS  and  user 
personnel 

22)  Organizational  level  of  top 
computer  executive  

23)  Source  of  origination  of  pro- 
ject (MIS  staff  or  user)  .  .  . 

24)  Participation  by  operating 
management  in  design,  formal 
approval  of  specifications, 
and  continual  review  of  pro- 
ject   
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Standard  factor:   "Performance  standards  employed 
for  analysts  and  nroBrammers" 


for  analysts  and  programmers' 


Importance  compared  to  standard: 

somewhat    about   somewhat   much 
lea*       same     more     more 


25)  Operating  management  conducts 
periodic  management  audit  of 
MIS  function.  (Evaluation  of 
effectiveness  for  users)  .  .  , 

26)  Utilize  existing  data  base 
versus  constructing  or 
greatly  modifying  one 


27)  Short-term,  minor  project 
versus  large,  complex  project. 

28)  Planning  and  accounting  for 
all  resources  throughout 
project  development  


29)  Utiliration  of  a  formal  time- 
scheduling  technique  such  as 
PERT  for  project  development  . 

30)  Use  of  a  formalised  and 
regular  reporting  structure 
on  project  progress 

31)  Ratio  of  computer  hardware 
investment  to  total  sales 

or  operating  budget 

32)  Overall  sire  of  organization 
systems  staff 

33)  Low  degree  of  overall  organi- 
zational change 

34)  High  centralization  of  organi- 
zational MIS  activities.  .  .  . 
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Please  list  any  other  factors  which  you  believe  are  important  to  the 
success  of  a  MIS  project. 
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APPENDIX   B 

CC  QUESTIONNAIRE  FOR  STUDY  OF 

1 SUCCESS/FAILURE  CORRELATES   IN  MIS 

2  PR0GR,U2iING  PROJECTS 


I.   Environmental  Factors 

A.   General  Information  -  Company  or  Division 

1.  Basic  Industry: 
a.   Manufacturing 

(1)  Consumer 

(2)  Industrial 

3  b.  Wholesale,  Retail  Trade 

c.  Transportation 

d.  Financial 

e.  Utility 

f   Ot^er    (Indlrsf  rm^ve) 

2.  If  Manufacturing,  Primary  Production  Technology 

a.  Unit  Job /Small  Batch 
4 

b.  Assembly  Line/Mass  Production 

c.  Continuous  Process 

3.  Company  or  Relevant  Division  Size  Data 
5-11             a.   Total  Assets 

12-18  b.  Total  Sales 

19-23  c.  Total  Employees 

24-26  d.  Number  of  Billed  Customers 

27-29  e.  Number  of  Products 

30-31  f.  Number  of  Manufacturing  Locations 

32-33  g.  Number  of  Distribution  Points 
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A.  Have  there  been  any  significant  organizational  changes  in 

last  three  years?   (i.e.,  mergers,  acquisitions, etc.) 

34  (Circle  one) 

Several 
No  signifi-             Minor                   Extensive 
cant  changes Changes Changes 

5         4  3  2  1 

5.  What  was  the  ratio  of  computer  hardware  investment  to  last 

35-36  year's  total  sales  or  other  gross  income  measure?   (If 

rented,  monthly  rental  x  40)  


39 


6.  What  proportion  of  the  operating  expenses  of  the  information 
37-38             systems  function  was  accounted  for  by  computer  hardware  last 
year?  


7.  Centralization  of  organization's  information  system  activities. 

a.   If  a  new  MIS  application  is  proposed,  which  of  the 

following  statements  best  describes  how  the  organization 

accomplishes  the  design,  analysis  and  programming 

activities  for  that  project?   (Check  one) 

All  design,  analysis,  and  programming 

tasks  are  performed  by  the  corporate 

MIS  staff  regardless  of  who  originated 

the  project.  5 


All  design,  analysis,  and  programming 
tasks  are  performed  by  the  corporate 
MIS  staff  only  if  the  project  involves 
organizational  segments  other  than  the 
one  proposing  the  project.   (I.E.,  If  a 
division  has  its  own  systems  staff,  and 
the  project  involves  only  that  division's 
information  needs,  the  division  will  develop 
its  own  programs  within  corporate  guidelines.) 

General  system  design  always  done  at  the 
corporate  level  but  analysis  and  programming 
performed  by  the  organizational  segment  that 
wants  the  project  accomplished. 
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The  corporate  level  MIS  staff  prescribes 
standards  and  procedures  for  project 
development,  approves  or  disapproves 
project  proposals,  and  evaluates  MIS 
progress  and  performance.  Corporate 
3taff  does  not  do  the  analysis  and 
programming. 

The  corporate  MIS  staff  provides  guidance, 
coordination,  and  assistance  to  organiza- 
tional segments  but  all  design,  analysis 
and  programming  performed  by  the  organiza- 
tional segment. 


8.   Level  of  the  top  computer  executive: 

a.  What  is  the  organizational  location  of  the  information 
systems  function: 
A  separate  function  reporting  directly  to  top  management 


A  separate  function  reporting  directly  to  administrative 
40  VP  or  other  similar  executive.  


An  organizational  component  of  some  other  staff 
function  such  as  the  Controller. 

An  organizational  component  of  a  line  function 
(specify  line  function) 


b.   How  many  hierarchical  levels  are  there  between  the 
manager  of  the  information  systems  function  and  the 
41  top  operating  executive?   (Circle  one) 

0          1          2          3          4  or  more 
42-44        9.  How  many  personnel  are  employed  by  the  company  in  the  systems 
design,  analysis,  and  programming  activities?  
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II.   Project  Information 

A.   Provided  by  Project  Leader  or  Information  Systems  iianager 

1.  \/ho  originated  this  project? 

User  department  A 

Top  level  management       3 

Information  systems  staff  2 

45  Other  (specify)  1 

2.  Wature  of  the  project; 

a.  What  functional  area  of  the  organization  was  the 

primary  beneficiary  of  the  project?  

(If  the  project  had  several  primary  beneficiaries, 
list  them  all.) 

b.  Project  leader's  brief  description  of  the  nature  of 
the  project. 

c.  Project  completion  date:  


3.   Objectives  of  the  Project:   (Check  most  appropriate  statement) 

a.  Specific  measurable  objectives  were  laid  down  in 

writing  at  the  outset  of  the  project.  < 

b.  Specific  but  non-measurable  objectives  were  laid 

down  in  writing  at  the  outset  of  the  project.      ! 

c.  General  but  clear  objectives  were  laid  down  in 

writing  at  the  outset  of  the  project.  i 

d.  General  objectives  were  agreed  upon  by 

key  parties  involved  at  outset  of  project  but 
A6  not  committed  to  writing.  '. 

e.  Objectives  of  the  project  were  rather  vague  at  its 
initiation  but  were  more  fully  developed  later  as 

the  project  progressed.  2 

f.  Project  objectives  were  not  quite  clear  to  those 

engaged  in  the  project.  1 


47 
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4.      Project  leader's  rating  of  the  relative  complexity  of  the 

project:      (Check  one) 

The  most   complex  project  I  have  been  involved  with 

in  this  organization  5 

One  of   the  more  complex  projects  we  have  undertaken  4 

About  average   complexity  3 

Less   cotiplex   than  many  of   the  organization's   MIS 

projects  2 


One  of  the  least  complex  I  US  projects  we  have  undertaken 1 

5.   Was  a  project  team  composed  of  user  personnel  and  information 
48  systems  staff  utilized  for  this  project?     Yes    1 


No     0 

If  yes: 

a.  Were  user  personnel  full  time  or  part  time  on  the 

49  project?  Full  tic.e  1 

Part  time  0 

b.  What  proportion  of  the  project  team  was  made  up  of 

50-51  user  personnel?  


c.   Of  the  total  man  months  devoted  to  the  project,  what 
proportion  were  accounted  for  by  user  personnel? 


52-53  d.  When  the  design  and  programming  effort  was  completed 

were  the  user  personnel  responsible  for  implementing 
54  the  project  in  their  respective  organizational  areas? 

Yes  

Wo 


e.   Did  the  user  personnel  who  worked  on  the  project  team 

55  carry  on  the  liaison  with  the  information  systems  staff 

after  implementation? 

Yes  1 

Uo  0 
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f.  '.Jhile  on  tlie  project  team,  to  whom  did  user 
personnel  report  as  their  superior? 

56  Project  leader         3 

Information  system  mgr.  2 

Regular  superior       1 

g.  Ky  whom  were  user  personnel  evaluated? 

Project  leader        3 

57  Information  system  mgr. 2 

Regular  superior      1 

h.   In  the  opinion  of  the  project  leader,  how  instrumental 

were  user  members  in  contributing  to  project  success? 

(Check  one) 

Very  great  contribution  —  were  critical  to  success 5 


58 


llade  some  contribution ;   could  not  have  done  so 
well  without  them 


Their  presence  on  the  team  didn't  contribute  much; 
could  have  done  just  as  well  without  them         3 

Their  presence  on  the  tear,  had  a  slightly  negative 
influence,  we  had  to  train  them,  etc,,  slowed  us  dcvn 

2 

Their  presence  on  the  project   team  was  detrimental 

to  project   success;    created  conflict  1 


59-60  6.     How  many  people  worked  directly  on  the  project? 

61-62  Number  of  Analysts 

63-64  number  of  Programmers 

65-66  User  Representatives 

67  Consultants 

68-69  7.     How  many  months  elapsed   from  initiation  of  the  project 
to  implementation? 

70-72  8.     How  many  man  months  were  spent  on  the  project? 
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9.  Were  the  analysis  and  programming  functions  accomplished  by 

1  separate  groups  or  by  personnel  who  were  combination  analysts/ 

programmers?  Combination  1 

Separate  group.?? 0 

10.   Systems  experience  of  project  personnel: 

2-4  a.  How  many  months  experience  in  systems  work  has  the 

project  leader  had? 

b.  What  is  the  mean  length  of  systems  experience  (in 
5-6  months)  for  project  personnel,  including  user 

organization  members  it  applicable?  

c.  Provide  sar.e  information  as  for  9  (b)  above 

7-9  excluding  user  organization  project  members.       

d.  What  proportion  of  the  project  staff  had  two  or 
more  years  of  systens  experience  at  the  beginning 

10-11  of  the  project?  

e.  Project  leader's  appraisal  of  impact  that  prior 
systems  experience  of  project  staff  had  on  project 
success: 

high  experience  uas  critical  to  project  success.   5 

12  Experience  contributed  somewhat  to  project  success. 4 

Experience  in  systems  work  was  not  very  important 

one  way  or  the  other.  3 


Experience  had  mixed  effects;  the  hi^h  experience  of 
some  team  metbers  was  very  important,  but  was  offset 
somewhat  by  inexperience  of  other  team  -..embers 


On  balance,  the  lack  of  systems  experience  was 
detrimental  to  project  success. 

11.  Project  staff  turnover: 

a.   What  was  the  percentage  turnover  within  the  project 

staff  during  the  course  of  the  project? 
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(This  should  be  calculated  as  follows) 


,.,  ,,  A  /  D 

13-14  -5p —    where: 

A=the  number  of   Individuals   added   to   the   project 
staff   as   replacements    for   departed  members 

D=the  number  of  individuals  who  left  the  project 
staff  oefore  the  project's  implementation  for 
any  reason  besides  completion  of  all  assigned  tasks. 

?rthe  median  number  of  individuals  on  the  project  staff. 

b.  Project  leader's  appraisal  of  the  effect  of  personnel  turnover 

on  project  success.   (Circle  one) 

15  Very  Somewhat       Hade  no        Contributed 

detrimental    detrimental difference to  success 

A  3  2  1 

12.  Project  staff  educational  level: 

16-17  a.  What  proportion  of  the  project  staff  held  college 

degrees?  

b.  Ulittt  proportion  o£    the   projeec   stail   compieLeu  at. 

18-19  least  tvo  years  of  college  or  Junior  college?       

c.  '■That  was   the  mean  number  of  years   of  education  for 

20-21  the  project  staff?  

(12  years  for  high  school ;  A  years  for  college,  etc. 
Person  with  a  college  degree  would  have  16  years  of 
education) 

13.  Organizational  experience  of  project  staff: 

a.   What  was  the  mean  number  of  years  experience  in  this 
22-23  organization  for  project  staff  members  at  the  beginning 

of  the  project?  
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b.   What  proportion  of  the  project  staff  had  two  or  more 
24-25  years  of  experience  in  this  organization  at  the  beginning 

of  the  project?  

14.   Documentation. 


a.   Were  formal  documentation  standards  specified  for  or 

26  applicable  to  the  project?  Yes 

No 


b.  Project  leader's  appraisal  of  importance  of  the 

documentation  standards*      (Circle  one) 

27  Ilade  signifi- 

cant contribu-  Somewhat  Were  just 

tion   to  success useful make-work 

5  4  3  2  1 

c.  Project  leader's  appraisal  of  compliance  of  the 

project  staff  with  documentation  standards: 

Project  staff  followed  standards  completely;  each 
phsse  of  documentation  required  by  standards  was 
completed  at  the  specified  time  during  project 
development.  


Project  staff  followed  standards  for  the  most  part; 
however,  documentation  was  occasionally  lacking  in 
quality  and  timeliness. 

28  Tried  to  follow  standards  but  just  couldn't  keep  up 

with  it  and  get  the  programming  done,  too. 

Standards  were  used  for  guidance  mainly  not  followed 
religiously. 


Did  not  follow  standards;  each  project  member  did  what 
he  felt  was  required.  


d.  Row  much  time  and  effort  were  devoted  to  enforcing 

documentation  standards?   (Circle  one) 

29  Very  great  Moderate  Very  little 

time  &  effort time  I   effort time  &  effort 

5  4         3  2         1 

e.  Project   leader's  appraisal  of  quality  of  the  documentation: 
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30  Documentation  was  of  high  quality;  experienced  no 

problems  because  of  inadequate  documentation. 

Documentation  was  adequate  in  most  cases .  some 
problems  of  inadequate  documentation  did  arise 
but  they  were  very  minor. 


Documentation  problems  were  frequent  but  such  problems 

did  not  seriously  affect  the  success  of  the  project. 2 

Documentation  problems  were  serious;  had  a  detrimental 
effect  on  success  of  the  project.  1 

f.   If  documentation  standards  were  not  followed,  was  this 
a  result  primarily  of:   (Check  one) 

Unrealistic  standards        

Inadequate  standards        

31  Lack  of  time  to  document     

Lack  of  willingness  to 

document  

Skip  32-33  Other  (specify)  


15.     Participation  by  user  organization  management  in  design, 
formal  approval  of  specifications,   and  continual   review 
of  project: 

a.  What  was  the  degree  of  the  user  organization's  active 

participation  throughout  the  evolution  of  this  project? 

34  (Circle  one) 

They 
Very    High  Moderate  provided  what    Almost 

great   participation   amount    was  a9ked  for    none 
5        4  3  2  1 

b.  What  contribution  do  you  feel  user  participation  made  to 

the  success  of  this  project? 

User  participation  was  definitely  an  important  contributor 
to  project  success  5 

35  User  participation  probably  contributed  somewhat  to 

project  success  4 


User  participation  of  no  real  consequence  one  way  or 
the  other 


The  lack  of  meaningful  user  participation  probably 
detracted  somewhat  from  project  success. 
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The  lack  of  meaningful  ujer  participation  definitely 

had  a  detrimental  effect  on  project  success.  ! 

c.  How  would  you  evaluate  the  quality  of  the  communications 
between  your  staff  and  the  user  organization  on  this 

36  project?   (Circle  one) 

Highly    constructive  Somewhat 

and    effective Gc.od      Aiequate      in.-?.''jqua  te        Poor 

5  4  3  2  1 

d.  How  clearly  were   user   organizations'    detailed    requirements 
and   desires    specified    at    the  outset  of    the   project? 
(Circle   one) 

37  Very   clearly  Generally  Not    too 

and    explicitly        clear Adequate        cle.ir Very   Va^ue 

5  4  3  ?  1 

e.  Was  management  at  the  top  levels  of  the  user  organization 
interested  in  the  project  and  in  directing  its  evolution? 

33  (Circle  one) 

Extremely  interested       Interested    Disinterested 

and  active  pnr  t  ic  ip<T  t  icn    but  passive    and  p-:isive 
5  4       3^1 

f.  Were  user  organizations'  desires  for  products  of  the 
project  incorporated  into  the  project  as  it  developed  even  if 

39  doing  so  entailed  changes  to  what  had  already  been  accomplished? 

(Circle  one) 

Always Usual  ly Hal  f  the  time Seldom Never 

Skip  40-41  5  4  3  2  1 

16.   VTiich  statement  most  nearly  matches  the  situation  that  existed 
for  project  control  on  this  project? 

There  was  a  formalized  reporting  system  which  required 

that  each  project  member  report  to  the  project  leader 

hi3  progress  against  assigned  or  planned  tasks  at  least 

every  month.  5 


42  No  formalized  reporting  system  was  required  by  the 

organization  but  the  project  leader  instituted  his  own; 
each  project  member  reported  his  progress  against 
assigned  or  planned  tasks  to  the  project  leader  at 
least  every  month. 
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There  was  no  regular  progress  reporting  required  of 
project  members,  However,  the  project  leader  was  in 
frequent  touch  with  all  project  rembers  and  thus  kept 
the  project  status  and  progress  records  himself. 

There  was  no  regular  progress  reporting  required  of 
project  members.  Project  leader  either  occasionally 
asked  project  members  how  they  were  progressing  or 
left  it  to  project  members  to  inform  him  if  falling 
behind  schedule. 

There  was  no  task  schedule  set  up  against  \;hich  to 
measure  progress  of  project  staff  members. 

A3       17.  What  programming  language  was  used  fcr  this  project?_ 


18.  Were  there  operating  system  or  other  generalized  software 

44  problems?   (Circle  one) 

No  problems Idnor  problems Ilajor  problems 

3  2  1 

B.   Provided  by  User  Personnel 

1.   User  organization  participation  in  the  projp.ct: 

45-46  a.  How  clearly  were  your  organization's  detailed 

requirements  and  desires  specified  at  the  outset 
of  the  project?   (Circle  one) 

Very  clearly    Generally  Not  too   Very 

and  explicitly  clear Adequately  clear vague 

5  4  3  2      1 

b.  What  was  the  degree  of  your  organization's  active 
participation  throughout  the  evolution  of  this 
47-48  project?   (Circle  one) 
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It  was  our  Provided 

project:  High  par-         Moderate     what  was       Alnost 

very  great  ticlpatlon       amount  asked   for    none 


5  4  3  2  1 

c.  Did  you  desire  to  have  more  of  a  voice  in  specifying 
and  approving  the  products  of  the  project  than  you 

49-50  actually  did  have? 

Couldn't 

have  had  Wanted  souiewhat        Definitely  did  not 

more  voice more  of  a  voice have  enough  voice 

5         4  3  2         1 

d.  How  would  you  evaluate  the  quality  of  the  communica- 
tions between  your  organization  and  the  systems  staff 

51-52  on  this  project?   (Circle  one) 

Highly  constructive  Somewhat 

and  effective Good   Adequate    inadequate   Poor 

5  4         3  2         1 

e.  Do  you  feel  this  project  resulted  in  what  yuu  wanted 
or  v'c.ct   che  staff  gave  you?   (Circle  one) 

53-54  Exactly  what      About  half       What  the 

we  wanted and  half staff  gave  us 

5         4     3        2       1 

f .  Did  you  make  formal  arrangements  for  liaison  between 
55  the  project  staff  and  your  personnel? 

Yes     1 

No  0 


Yes  1 


56  g.   Did  you  assign  any  person  or  persons  in  your 

organization  responsibility  for  coordinating  the 
project  in  your  organization? 

No     0 

57  h.   If  answer  to  (e)  was  yes,  did  you  relieve 

this  person  (persons)  of  normal  duties? 

Yes  1 

No        0 
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III .   Success  Criteria 

(A  project  is  considered  cooplete  when  turned  over  to  computer 
operations) 

A.   Time 

US  manager  or  project  leader 

1.   a.   Was  the  project  completed  within  the  calendar 

61  time  originally  allocated  for  it? 

Yes  

No 


68-70 


b.   If  completed  on  time,  were  additional  staff 
62  resources,  over  what  originally  were  planned, 

required  to  do  so? 


Yes 

No  " 


c.  If  answer  to  (b)  is  yes,  what  percentage  of 
the  total  project  effort  in  nan  months  vas 

63-64  accounted  for  by  these  additional  resources? 

d.  If  corjplctcd  on  schedule  vie  un<T!tic"'.'\'1t'?d 
65                      overtime  required  to  do  so? 


Yes 

No  " 


e.   If  answer  to  (d)  is  yes,  what  percentage  of 
66-67  total  project  time  in  man  months  was  overtime? 


2.  What  were  the  actual  nan  months  required  to  complete  the 
project  as  a  percentage  of  man  months  originally  allocated 

to  the  project?  

71-73  3.  What  was  the  actual  project  completion  time  as  a 

percentage  of  allocated  time?   (elapsed  time)      

4.   a.   Did  the  project  leader  believe  the  time  originally 

74  allocated  for  the  project  was  realistic? 

Yes 1 

No  ~      C 


b.   If  the  answer  to  (a)  is  no,  what  did  the  project 


-220- 


Card  03 


leader  believe  to  be  a  realistic  tine  period  for 
completion  of  the  project  (as  a  percentage  of  the 
time  actually  allocated  -  e.g.,  120%)         


5.  If  the  tine  schedule  v;as  not  net;  uhat  was  the  primary 
reason  given  by  the  project  leaaer? 

(Answer  this  question  on  the  back  of  preceding  page.) 

6.  Relative  to  past  experience,  how  successful  does  the 
project  leader  feel  this  project  was  in  terms  of  time 

75  to  complete  it?   (Circle  one) 

Highly      Above  Below     Very 

successful   average   Average   average   poor 
5  4         3         2        1 


B .   Cos  t : 


IIIS  lianager  or  Project  Leader 

1.  Was  the  project  completed  "ithin  the  original  cost 

1  budget  for  the  project?  Yes 1 

Ho  0 

2.  What  was  the  actual  project  cost  as  a  percentage  of 

2-4  budgeted  cost?  


3.  If  the  project  exceeded  budget,  what  was  the  primary 
reason  for  this  in  the  opinion  of  the  project  leader? 

4.  Relative  to  past  experience,  hew  successful  does  the 

project  leader  feel  this  project  was  in  terms  of  cost? 

(Circle  one) 

Highly       Above  Below      Very 

successful   average    Average   average poor 

5        4  3       2         1 
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C.   User  Perceptions  of  Success  of  Project: 
For  each  prlcu-ry  user  organization- 
For  two  levels  of  management  in  the  user  organization 

1.  How  closely  do  the  products  of  the  project  match  what  you 

wanted  and  expected  from  the  project?   (Circle  one) 

6-7  Got  even 

more   than  Good  Satisfactory       Marginal       Poor 

expected  match  match match match 

5  A  3  2  1 

2.  Do  you  believe  the  originally  stated  objectives  for  the 
project  were,  or  are  in  the  process  of  being,  satisfied? 

8-9  (Circle  one) 

For  the     Not      Hot  very   Definitely 

Definitely    most  part   certain   well not 

5  4         3         2        1 

3.  Overall,  how  satisfied  are  you  with  the  products  of  the 

project?  (Circle  one) 

Highly     Well       Products     Products 
10-11  pleased    satisfied   acceptable   uar^inal   Dissatisfied 

5        4  3  2         1 

4.  Have  there  been  implementation  problems  associated  with 

this  project  in  your  organization?   (Circle  one) 

Very 
»  Very  minor   Moderate  Considerable  serious 

12-13  No  problems   problems problems   difficulties  problems 

5  4  3  2         1 

5.  How  well  has  this  project  been  accepted  by  your  organization? 
(Circle  one) 
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Very 
Good       Satisfactory  Marginal    negat- 
14-15  Enthusiastically   acceptance  acceptance    acceptance  ively 

5  4  3  2  1 

6.  What  do  you  think  should  have  been  done  to  improve  the 

effectiveness  and  value  of  the  project  with  respect  to 

your  information  needs? 

16-17         7.   Do  you  feel  this  project  is  cocpleted?  Yes I 

No  0 

Skip  18-21 
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D.   Project  Leader's  perception  of  Success  of  Project: 

1.  Do  you  believe  the  originally  stated  objectives  for  the 
project  were,  or  are  in  the  process  of  being,  satisfied? 

22  (Circle  one) 

For  the    Not      Hot  very 

Definitely  most  part  certain  well Ho 

5        A         3        2       1 

2.  Overall,  how  successful  do  you  think  this  project  was? 
(Circle  one) 

23  Extremely   Very        Moderately  iJot 

successful  successful  successful  certain  Marginal 

5         4  3         2    "     1 

3.  How  well  satisfied  do  you  feel  the  users  are  with  the 
products  of  this  project?   (Circie  one) 

Highly   Well  Somewhat 

24  pleased  satisfied  Satisfied  dissatisfied  dissatisfied 

5       4  3  2         1 

4.  VJere  there  implementation  problems  associated  with   this 

25  project?    (Circle  one) 

Very 
Very  minor     Moderate     Considerable     serious 

Ho  problems     problems  problems     problems problems 

Skip  26-27  ,5  4  3  2  1 
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E.   Computer  Operating  Cost; 

i'JS  Manager  or  Computer  Operations  Kanager 

1.  Has  this  project  caused  problems  for  computer  operations? 

20  (Circle  one) 

Very 
Very  minor  Moderate  Considerable   serious 
No  problems  problems   problems  difficulties   problems 
5         4         3  2  1 

2.  Would  you  say  the  costs  of  operation  for  this  project  are 

29  excessive?   (Circle  one) 

Definitely  Frobably  Not      Probably  Definitely 

not not certain  are are 

5       4         3         2        1 

3.  Are  you  running  this  project  less  frequently  than  the  users 
desire?   (Circle  one) 

30  Run  Whenever  Usually  run  Not      Run  much  less  Virtually 

desired      as  desired  certain  than  desired   never  run 
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LETTER  REQUESTING  PARTICIPATION  IN  STUDY 


MANAGEMENT  INFORMATION  SYSTEMS  RESEARCH  CENTER 

SCHOOL  OF  BUSINESS  ADMINISTRATION 

MINNEAPOLIS,  MINNESOTA  55455 

July  9,  1970 


Representative's  Name 

Position 

Firm 

Address 

Dear  Mr.  Representative: 

One  of  our  MIS  doctoral  students,  Dick  Powers,  Is  doing 
research  on  the  organizational  and  procedural  correlates 
of  success  with  MIS  projects.  Dick  has  reached  the  point 
where  he  desires  to  gather  data  on  twenty  MIS  projects  for 
a  statistical  analysis  of  several  hypotheses  he  has  set  up. 
He  hopes  to  collect  the  required  data  by  Interviewing 
selected  people  In  ten  associate  firms  (two  MIS  projects  per 
firm)  during  the  period  1  October  to  31  December,  1970. 

Within  the  next  several  days,  Dick  will  be  contacting  you 
to  arrange  an  appointment  of  about  thirty  minutes  duration 
to  explain  more  fully  the  research  he  is  doing,  and  to  discuss 
what  will  be  Involved  on  your  part  should  you  agree  to  par  tic  1« 
pate.   I  believe  the  research  is  most  worthwhile  and  hope  you 
will  elect  to  participate. 

Since  Dick  will  be  doing  all  of  the  data  collection  and 
analysis  himself,  it  is  appropriate  that  you  know  something 
about  his  qualifications  and  background.  To  this  end,  a 
brief  biographical  sketch  is  attached. 

Thank  you  for  your  continued  support  and  cooperation. 

Sincerely, 


Gary  W.  Dickson 
Associate  Professor 

GWD/jls 
Enclosure 
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Brief  Biographical  Sketch  of  Richard  F.  Powers 


Dick  Powers  is  currently  a  doctoral  student  majoring  in 
management  information  systems  at  the  University  of  Minnesota. 
He  entered  the  MIS  program  in  the  fall  of  1968  and  has  now 
completed  all  course  work  required  for  the  PhD.  Dick  is  a 
lieutenant  commander  in  the  Supply  Corps  of  the  U.S.  Navy, 
and  is  being  sponsored  in  his  doctoral  studies  by  the  Navy. 
He  holds  a  BA  degree  from  Rice  University,  1958,  and  an  MBA 
from  Michigan  State  University,  1965. 

Since  his  commissioning  as  an  ensign  in  1958,  Dick  has  held 
the  following  positions  up  to  the  time  he  entered  the  MIS 
program  at  Minnesota: 

1959-1960     Supply  and  Disbursing  Officer,  USS  MASSEY 

(DD-778) 
1961         Aide  to  the  Commanding  Officer,  Naval 

Supply  Center,  Norfolk,  Va. 
1962-1963     Director  of  Data  Processing,  Naval 

Supply  Center,  Norfolk,  Va. 

1964  Control  Division  Officer,  Royal  Canadian 
Naval  Supply  Depot,  Victoria,  British 
Columbia 

1965  MBA  program,  Michigan  State  University 
1966-1968           Director  of  Computer  Training,  Navy 

Supply  Corps  School,  Athens,  Ga. 
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BRIEF  DESCRIPTION  OF  STUDY 


MANAGEMENT  INFORMATION  SYSTEMS  RESEARCH  CENTER 

SCHOOL  OF  RUSINESS  ADMINISTRATION 

MINNEAPOLIS,  MINNESOTA  55455 


Research  on  the  Correlates  of  MIS  Project  Success 

Nature  of  the  Research; 

An  empirical  investigation  of  sixteen  organizational 
and  procedural  factors  hypothesized  to  be  correlates  of  success 
with  MIS  projects. 

Methodology; 

Study  two  MIS  projects  in  each  of  ten  organizations 
with  as  wide  representation  across  industries  as  possible. 

Criteria  of  Success  for  a  Project; 
Time 
Cost 

User  satisfaction 
Computer  operating  problems 

Data  Collection; 

By  structured  Interview  that  follows  a  pre- tea  ted 
questionnaire.  The  responses  are  anchored,  and  multiple 
scales  are  used  wherever  possible  to  improve  reliability 
of  the  results. 

For  each  project  selected  for  investigation  the 
following  individuals  will  be  interviewed  with  the 
interviews  requiring  approximately  the  amount  of  time 
shown: 


Individual 

Time 

Project  Leader 

2  Hours 

User  Management, 

Level  1 

%  Hour 

User  Management, 

Level  2 
Total 

\   Hour 

Project 

3  Hours 

For  Two 

Projects 

6  Hours 

In  addition  to  the  above  interviews  for  each  of  two 
projects  in  each  organization,  Interviews  will  be  conducted 
with  the  following  individuals  to  collect  computer  operat- 
ing Information  or  information  not  specifically  related  to 
one  project: 
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MISRC  Associate 

Manager  of  MIS 

Manager  of  Computer  Operations 


h 

Hour 

h 

Hour 

h 

Hour 

\\   Hours 


Total  Interviewing  Time 

per  Organization     1\   Hours 

Data  Analysis: 

All  data  collected  will  be  analyzed  statistically 
using  non-parametric  techniques  such  as  X  .   Qualitative 
analysis  of  various  moderator  variables  will  also  be  made, 

Expected  Results  of  the  Research; 

1.  Confirmation  or  rejection  of  the  numerous 
prescriptions  appearing  in  information  systems  literature. 

2.  A  ranking  or  other  classification  of  those 
organizational  and  procedural  factors  related  to  MIS  project 
success  in  the  operational,  real-world  environment.   This 
ranking  or  classification  will  have  more  general  meaning  for 
MIS  management  than  previous  case  studies. 

3.  The  manager  should  be  able  to  judge  more 
accurately  which  organizational  and  procedural  factors  are 
most  critical  to  project  success  given  a  particular  set  of 
circumstances. 

Assurances  to  the  Participating  Organization: 

1.  Reported  results  of  the  research  will  be  made 
available  to  the  participant  company. 

2.  No  company  names  nor  non-public  financial 
information  will  be  reported  without  the  express  prior 
permission  of  the  company. 

3.  No  information  provided  by  an  individual  will 
be  divulged  to  any  other  person  or  be  attributed  to  the 
individual  in  the  report  without  the  express  prior  consent 
of  that  individual. 

4.  All  information  gathered  will  be  retained 
personally  by  the  researcher  at  all  times. 
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APPENDIX  E 
VARIABLES  INCLUDED  IN  THE  STUDY 


Variable  Variable  Source 

Number  Name  Page   Question(s) 


Variable  Description  and  Scoring 


ASSETS  206      3a 

Collected  as  a  measure  of  relative 
size  of  organizations  in  the  sample. 
Asset  data  were  not  available  for 
three  of  the  organizations. 

SALES  206      3b 

Total  sales  or  other  gross  revenue 
measure  for  the  past  fiscal  year. 
Not  available  for  two  of  the  organi- 
zations in  the  sample. 

EMPLOYEES  206      3c 

Number  of  employees  in  domestic 
operations  of  each  organization. 
The  only  measure  of  organization 
size  which  was  available  for  all 
organizations  in  the  sample. 

CUSTOMERS  206      3d 

Total  billed  customers.   Collected 
as  a  measure  of  relative  organiza- 
tion size.   Not  available  for  two 
organizations. 

ORGANIZATION  CHANGE  207      A 

Degree  of  organizational  change 
during  past  three  years;  scaled 
from  "no  change"  at  high  end  (5) 
to  "extensive  change"  at  low  end 
(1).   Pertains  to  changes  affect- 
ing the  whole  organization  as 
opposed  to  intra functional  realign- 
ments. 
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HARDWARE  INVESTMENT/SALES  207       5 

Measure  of  investment  in  computing 
hardware  and  supporting  equipment 
relative  to  total  revenue  for  past 
fiscal  year.   If  owned,  equipment 
valued  at  replacement  cost  or 
current  list  price  for  identical 
or  comparable  equipment.   If  rented, 
monthly  rental  price  multiplied  by  40. 

HARDWARE  EXPENSE/BUDGET  207       6 

Represents  the  proportion  of  last 
fiscal  year's  budget  for  the 
information  systems  function  that 
was  accounted  for  by  hardware  cost 
(depreciation  and/or  rental). 

CENTRALIZATION  207       7 

Centralization  of  analysis,  design 
and  programming  activities  in  the 
organization.   Scaled  from  complete 
centralization  (5)  to  decentraliza- 
tion (1).   This  variable  was  not 
adequately  tested  by  the  present 
sample;  eight  organizations  were 
scrred  at  5  level  and  remaining  two 
were  scored  at  4  level.   The  latter 
two  were  not  at  the  5  level  solely 
because  of  small  operations  research 
groups  which  did  their  own  design 
and  programming.   For  this  reason, 
Variable  8  is  not  tabled  with  other 
variables  in  the  study. 

LEVEL  OF  INFORMATION  SYSTEMS  MANAGER    208       8b 

Number  of  hierarchical  levels  between 
the  manager  of  the  information  systems 
function  and  the  too  operating  execu- 
tive of  the  organization.   Scaled 
from  no  intervening  levels  (0)  to 
four  or  more  intervening  levels  (4). 
The  lower  the  score  on  this  variable, 
the  higher  the  information  systems 
function  in  the  organization. 
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10  NUMBER  IN  SYSTEMS  &  PROGRAMMING  208 

This  variable,  the  total  number  of 
personnel  engaged  in  systems  analy- 
sis, design,  and  programming  activi- 
ties within  the  organization,  is 
used  as  the  numerator  for  computing 
the  value  of  Variable  61. 

11  ORIGINATOR  209 

This  variable  identifies  the  primary 
originator  of  a  project  being  studied. 
It  is  scaled  from  user  origination 
(4)  to  information  systems  staff 
origination  (2);  there  were  no 
projects  reported  as  "other"  (1). 
In  terms  of  rankings,  high  rank  is 
given  to  user  originated  projects; 
middle  rank  to  those  of  top  manage- 
ment origin;  and  low  rank  to  systems 
staff  originated  projects. 

12  MEASURABLE   PROJECT   OBJECTIVES  209 

The  project  leader's  perception  of 
nature  and  clarity  of  project  objec- 
tives.  Scaled  from  highly  specific 
and  measurable  objectives  in  writ- 
ing (6)  to  very  vague  objectives  (1). 

13  COMPLEXITY  210 

Project  leader's  perception  of  the 
complexity  of  this  project  as  compared 
with  others  the  organization  has  under- 
taken.  The  higher  the  scale  value  for 
this  variable,  the  greater  the  complex- 
ity, with  most  complex  scored  5  and 
least  complex  scored  1. 

14  PROJECT  TEAM  210 

Dichotomous  variable  scored  1  if  a 
project  team  including  user  personnel 
developed  the  project,  and  scored  0 
if  there  was  not  a  project  team  com- 
prised, in  part,  of  user  personnel. 
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15  USER  MAN  MONTHS /TOTAL  MAN  MONTHS        210       5c 

The  proportion  of  man  months  spent  on 
the  project  accounted  for  by  user 
personnel.   Variable  15  value  will 
be  0  for  all  projects  where  value  of 
Variable  14  is  0,  indicating  a  project 
team  including  users  was  not  employed. 

16  USER  CONTRIBUTION  TO  TEAM  211       5h 

Project   leader's   perception  of   the 
contribution  of  user  members   of    the 
project   team   to  project  success. 
A  value   for  only   13  projects    that 
used    team.      Not   included    in   the 
statistical   analysis. 

17  NUMBER  ON  PROJECT  211       6 

Total  number  of  personnel,  including 
users  and  consultants,  if  any,  who 
worked  directly  on  the  project  at 
any  time. 

18  NUMBER  OF  ANALYSTS  211       6 

Number  of  analysts  who  worked  on 
project.   Not  included  in  the 
statistical  analysis. 

19  NUMBER  OF  PROGRAMMERS  211      6 

Number  of  programmers  who  worked  on 
project.   Not  included  in  the 
statistical  analysis. 

20  NUMBER  OF  USERS  211      6 

Number  of  user  personnel,  if  any, 
who  worked  directly  on  project. 
Not  included  in  the  statistical 
analysis . 

21  ELAPSED  MONTHS  211       7 

Number  of  months  included  in  period 
from  formal  start  of  project  to  pro- 
ject completion;  project  completion 
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defined  as  date  when  project  turned 
over  to  computer  operations  for 
production  running. 

22  MAN  MONTHS  211      8 

Number  of  man  months  spent  on  project 
by  all  personnel  directly  involved 
with  project,  including  analysts, 
programmers,  user  members  of  project 
team,  and  consultants,  if  any. 

23  COMBINATION  ANALYST/ PROGRAMMER  212       9 

Dichotomous  variable  scored  1  if 
combination  analyst/programmers 
were  used  on  the  project,  and 
scored  0  if  the  analysis  and 
programming  tasks  were  performed  by 
separate  individuals  or  groups. 

24  SYSTEMS  EXPERIENCE  OF  PROJECT  LEADER    212      10a 

Total  number  of  months  systems 
experience  the  project  leader 
possessed  at  the  start  of  the 
project.   Systems  experience 
defined  to  include  programming  or 
systems  work  performed  outside  of 
a  data  processing  organization,  such 
as  methods  and  time  studies.   This 
variable  not  used  in  the  statistical 
analysis;  Variable  27  used  as  the 
systems  experience  variable  in  all 
reported  cross-tabulations.   Variable 
24  was  highly  correlated  with  Variable 
27,  the  tau  value  being  .418,  which 
is  significant  at  the  .014  level 
(two-tailed). 

25  MEAN  SYSTEMS  EXPERIENCE  OF  PROJECT 

PERSONNEL  INCLUDING  USERS  212      10b 

Mean  number  of  months  systems  exper- 
ience for  entire  project  team, 
including  user  personnel  and  consul- 
tants, if  any.   Comments  under  Variable 
24  concerning  definition  of  systems 
experience  and  tabulation  apply 
fully  to  Variable  25.   Tau  value  for 
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Variable  25  vs.  Variable  27  was 
.596,  which  is  significant  at  the 
.0006  level  (two-tailed). 

26  MEAN  SYSTEMS  EXPERIENCE  OF  PROJECT 

PERSONNEL  FROM  MIS  STAFF  ONLY  212       10c 

Mean  number  of  months  systems  experi- 
ence for  those  personnel  from  the 
information  systems  function  who 
worked  on  the  project.   This  value 
the  same  as  the  value  for  Variable 
25  for  those  seven  projects  where  a 
team  not  used.   Comments  under 
Variable  24  concerning  definition 
of  systems  experience  and  tabulation 
apply  fully  to  Variable  26.   Tau 
value  for  Variable  26  vs.  Variable 
27  was  .575,  which  is  significant 
at  the  .0004  level  (two-tailed). 

27  TWO  OR  MORE  YEARS  SYSTEMS  EXPERIENCE    212       lOd 

Proportion  of  the  project  staff, 
including  users  and  consultants,  if 
any,  with  two  or  more  years  systems 
experience.   Systems  experience 
defined  exactly  as  for  Variables 
24-26.   This  measure  of  systems 
experience  considered  to  contain  the 
least  distortion  from  extreme  values. 

28  IMPACT  OF  SYSTEMS  EXPERIENCE  212       10c 

Project   leader's   opinion  of   the 
impact   that  prior   systems    experience 
of  project  personnel  had  on  project 
success.      Scaled   from  highly  critical 
to  success    (5)    to  a   lack  of  systems 
experience  detrimental    to  success    (1). 
This  variable  not   tabled   in   the 
statistical   analysis. 

29  TURNOVER  21?       11a 

The  percentage  turnover  of  the  project 
staff.   Any  person  who  left  the 
project  staff  at  any  time  prior  to 
completion  of  the  project  was  considered 
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as  contributing  to  turnover  unless 
that  person's  assigned  tasks  had 
been  completed.   If  a  person  who 
left  the  project  prematurely  was 
replaced,  the  replacement  was  also 
considered  as  a  factor  in  the  numera- 
tor of  the  turnover  computation. 
The  denominator  of  the  turnover 
computation  was  the  "normal"  or  median 
number  of  personnel  on  the  project 
staff  multiplied  by  two. 

30  COLLEGE  DEGREE  213       12a 

Proportion  of  project  staff  that 
held  a  college  degree.   Although 
not  used  as  the  primary  measure  of 
the  level  of  education,  Variable  30 
did  make  a  contribution  to  the 
interpretation  of  certain  relation- 
ships, and  for  this  reason  it  has 
been  cross-tabulated  with  other 
variables  where  significant 
relationships  existed. 

31  TWO  YEARS  COLLEGE  213       12b 

Proportion  of  project  staff  that 
had  completed  at  least  two  years 
of  college.   This  proportion 
included  all  of  those  who  held 
college  degrees,  and  was,  therefore, 
very  strongly  associated  with 
Variable  30.   Since  Variable  31 
contributed  nothing  to  the  study 
which  was  not  picked  up  by  using 
Variables  30  and  32,  it  is  not 
tabled  in  the  statistical  analysis. 

32  MEAN  YEARS  FORMAL  EDUCATION  213       12c 

Mean  number  of  years  formal  educa- 
tion for  the  project  staff,  including 
users  and  consultants,  if  any.   This 
variable  used  as  primary  measure  of 
level  of  education  for  project  staff. 
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33  MEAN  YEARS  ORGANIZATION  EXPERIENCE       213       13a 

Mean  number  of  years  experience  in 
the  organization  (firm,  not  function) 
studied  for  project  staff  members. 
Not  included  in  the  statistical 
analysis  since  Variable  34  believed 
to  be  a  less  distorted  measure  of 
organization  experience. 

34  TWO  OR  MORE  YEARS  ORGANIZATION 

EXPERIENCE  214      13b 

Proportion  of  the  project  staff, 
including  users  and  consultants, 
if  any,  possessing  two  or  more 
years  experience  in  the  organiza- 
tion (firm,  not  function)  studied. 
This  variable  the  primary  measure 
of  organization  experience. 

35  DOCUMENTATION  STANDARDS  214      14a 

Dichotomous  variable  scored  1  if 
documentation  standards  were 
prescribed  for  the  project,  and 
scored  0  if  such  standards  were 
not  prescribed. 

36  QUALITY  OF  DOCUMENTATION  214      14e 

Project  leader's  appraisal  of  the 
quality  of  documentation  for  the 
project.   Scaled  from  high  quality 
(4)  to  serious  problems  with 
documentation  (1). 

37  STANDARDS  APPRAISAL  214      14b 

14c 
The  sum  of  two  separate  perceptions 
by  the  project  leader;  how  important 
he  felt  the  documentation  standards 
were  plus  how  well  the  project  staff 
complied  with  the  standards.   Since 
Variable  37  made  no  contribution  to 
the  analysis  of  documentation  stan- 
dards or  quality,  it  is  not  tabled. 
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38  SPECIFICITY  OF  USER  REQUIREMENTS— PL    216       15d 

Project   leader's   perception  of   the 
clarity  and  specificity  of  user 
requirements   and  desires    at   the 
beginning  of    the  project.      Scaled 
from  very  clear  and  specific    (5) 
to  very  vague    (1). 

39  USER  MANAGEMENT  INTEREST  216       15e 

Project  leader's  perception  of  the 
interest  and  participation  of  top 
level  user  management  in  the  develop- 
ment of  the  project.   Scaled  from 
very  active  participation  (5)  to 
disinterested  (1). 

40  CHANGED  AS  REQUESTED  216       15f 

Project  leader's  perception  of  the 
response  by  the  project  staff  to 
user  requests  for  changes  in  any 
aspect  of  the  project  during  the 
development  period  for  the  project. 
Such  changes  may  have  entailed 
merely  output  format  modifications 
or  may  have  required  major  logic 
alterations.   Scaled  from  always 
made  requested  changes  (5)  to 
never  made  such  changes  (1).   If 
no  changes  of  any  kind  were  requested 
by  the  user,  scored  5. 

41  USER  PARTICIPATION— PL  215      15a 

15b 
Project  leader's  perception  of  the  15c 

level  of  user  participation  in  the 
project.   An  aggregate  variable 
representing  the  sum  of  three  separ- 
ate items.   Scaled  from  very  great 
participation  (15)  to  very  little  (3). 

42  PROJECT  CONTROL  216       16 

Project  leader's  rating  of  the  type 
and  frequency  of  progress  reporting 
for  the  project.   Scaled  from  formal- 
ized, monthly  progress  reporting  (5) 
to  no  progress  reporting  (1). 
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43  HIGH  LEVEL  PROGRAMMING  LANGUAGE         217       17 

Projects  programmed  in  COBOL  were 
scored  3;  those  programmed  in  FORTRAN 
were  scored  2;  and  those  programmed 
in  some  assembly  language  were  scored  1. 
For  the  purposes  of  this  study,  COBOL 
was  considered  a  higher  level  language 
than  FORTRAN. 

44  GENERALIZED  SOFTWARE  PROBLEMS  217       18 

Project  leader's  perception  of  the 
existence  and  seriousness  of  operating 
system  or  other  generalized  software 
problems  with  respect  to  this  project. 
Scaled  from  no  problems  (3)  to  major 
problems  (1). 

45  SPECIFICITY  OF  USER  REQUIREMENTS- 
USER  217        la 

Users'  perception  of  the  clarity  and 
specificity  of  user  requirements  and 
desires  at  the  beginning  of  the 
project.   Scaled  from  very  clear  and 
explicit  (50)  to  very  vague  (10).* 

46  USER  PARTICIPATION— USER 

Users'  perception  of  the  level  of 

user  participation  in  the  project. 

An  aggregate  variable  representing 

the  sum  of  four  separate  items. 

Scaled  from  very  great  participation 

(200)  to  very  little  participation  (40).* 

47  ON  TIME  219        la 

A  dichotomous  variable  scored  1  if 
the  project  was  completed  within  the 
calendar  time  originally  estimated, 
and  scored  0  otherwise.   Variable  47 
not  used  in  cross -tabulations  since 
Variable  48  was  the  primary  time 
success  variable  used  in  the  study. 
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48  ACTUAL  TIME/ESTIMATED  TIME  219 

The  ratio  of  actual  calendar  months 
spent  on  the  rpoject  over  calendar 
months  estimated  for  the  project. 
This  variable  is  the  criterion 
variable  for  project  success  in 
terms  of  time. 

49  TIME  SUCCESS— PL  220 

Project  leader's  opinion  of  the 
success  of  this  project,  in  terms 
of  time  to  complete  it,  relative 
to  his  experience  with  actual  time/ 
estimated  time  for  other  projects. 
Scaled  from  highly  successful  (5) 
to  very  poor  (1). 

50  WITHIN  BUDGET  220 

A  dichotomous  variable  scored  1 
if  the  project  was  completed  with- 
in the  original  cost  budget  for 
the  project,  and  scored  0  other- 
wise.  Variable  50  not  used  in 
cross -tabulations  since  Variable 
51  was  the  primary  cost  success 
variable  used  in  the  study. 

51  ACTUAL  COST/ESTIMATED  COST  220 

The  ratio  of  actual  cost  of  the 
project  over  estimated  or  budgeted 
cost  for  the  project.   This  variable 
is  the  criterion  variable  for  success 
in  terms  of  cost.   Where  cost  data 
were  not  available,  this  ratio 
computed  based  on  actual  and  estimated 
man  months  for  the  project,  along 
with  an  estimated  factor  for  computer 
test  time. 

52  COST  SUCCESS— PL  220 

Project   leader's   opinion  of   the 
success   of    this   project,    in   terms 
of  cost,   relative   to  his   experience 
with  actual   cost/estimated   cost   for 
other   projects.      Scaled    from  highly 
successful    (5)    to  very  poor    (1). 
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53  IMPLEMENTATION  PROBLEMS— USER  221       4 

Users'  perception  of  the  problems 
experienced  in  implementing  the 
project  in  their  organizations. 
Scaled  from  no  problems  (50)  to 
very  serious  problems  (10).* 
Although  this  variable  is  actually 
a  component  of  Variable  54,  it  is 
also  treated  separately  for  compara- 
tive purposes. 

54  USER  SATISFACTION— USER  221       1 

2 
Users'  perception  of  the  success  of  3 

the  project  in  terms  of  five  sepa-  4 

rate  factors  which  were  summed  to  5 

derive  one  overall  measure  of  user 
satisfaction.   Scaled  from  high 
user  satisfaction  (250)  to  low 
user  satisfaction  (50).*  This 
variable  is  the  criterion  variable 
for  success  in  terms  of  user  satis- 
faction. 

55  SATISFACTION  OF  OBJECTIVES— PL  223       1 

Project  leader's  perception  of  the 
degree  to  which  project  objectives 
were  satisfied.   Scaled  from  defin- 
itely satisfied  (5)  to  not  satisfied 
(1). 

56  PROJECT  SUCCESS— PL  223       2 

Project  leader's  global  perception 
of  the  overall  success  of  the 
project.   Scaled  from  extremely 
successful  (5)  to  marginally 
successful  (1). 

57  USER  SATISFACTION— PL  223       3 

Project   leader's   perception  of 
user  satisfaction  with    the 
products   of    the  project.      Scaled 
from  users   highly  pleased    (5) 
to  users   dissatisfied    (1). 
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58  IMPLEMENTATION  PROBLEMS- -PL  223       4 

Project  leader's  perception  of  the 
problems  encountered  in  implementing 
the  project.   Scaled  from  no  problems 
(5)  to  very  serious  problems  (1). 

59  COMPUTER  OPERATIONS  PROBLEMS  224       1 

Perception  of  the  manager  of  computer 
operations,  or  his  superior  in  some 
cases,  of  the  problems  caused  by  the 
project  for  computer  operations. 
Scaled  from  no  problems  (5)  to  very 
serious  problems  (1).   This  variable 
is  the  primary  criterion  for  project 
success  in  terms  of  computer  operations. 

60  COMPUTER  OPERATIONS  COST  224       2 

Perception  of  the  manager  of  computer 
operations,  or  his  superior  in  some 
cases,  of  the  computer  operating 
cost  for  the  project.   Scaled  from 
cost  definitely  not  excessive  (5)  to 
cost  definitely  excessive  (1). 

61  SYSTEMS  STAFF /TOTAL  EMPLOYEES  Computed 

from 
A  measure  of  the  relative  size  Variables  3 

of  the  organization's  systems  and  10 

staff  (analysts  and  programmers). 
Computed  by  dividing  the  total 
number  of  analysts  and  programmers 
(Variable  10)  by  the  total  number 
of  domestic  employees  of  the 
organization  (Variable  3). 


*  Note:   All  variables  representing  users'  perceptions 
were  scaled  up  by  10   to  avoid  using  fractions 
where  there  were  two  or  more  responses  per 
item  which  were  averaged  to  derive  one  value 
for  that  item.   Such  scaling,  of  course,  had 
no  bearing  on  the  relative  rankings  which  were 
used  in  the  analyses. 
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APPENDIX  F 


COMPUTING  FORMULAS  USED 


1 


The  following  procedures  and  formulas  were  used  in 
computing  the  values  of  tau-B,  tau-C,  S,  variance  of  S,  and 
the  normal  deviate  of  S: 

1)  All  observations  on  two  variables  being  cross- 
classified  were  entered  into  a  naturally  ordered 
table  of  the  following  type: 

j"l  to  c  columns 


J 


175 

150 

i=l  to  r  130 

rows    92 

58 

47 

40 


5 

4 

3 

2 

1 

1 

0 

0 

1 

2 

0 

1 

0 

2 

0 

3 

0 

0 

1 

0 

e 

tc. 

*M- 

nv, 


Ij 


i=l 


ni.=ZXU 
J  si 


In  the  above  example  case,  there  are  five  possible 
levels  for  one  variable,  and  these  are  represented  by 
the  columns.   There  are  seven  possible  values  for  the 
second  variable  and  these  are  represented  by  the  rows. 
Any  one  observation  of  the  total  N  observations  must 
fall  in  one  of  the  cells  of  the  resulting  array.   Thus, 


Most  of  what  is  presented  in  this  appendix  is  taken  from 
the  appendix  of  UMST  590,  the  computing  program  used  for  the 
data  analysis,  which,  in  turn,  is  from  Kendall  (1962). 
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the  entry   in  each  cell    is  merely   the  number   of 
observations    falling   in   that  cell,   or  having    the 
values   of  variables   one  and    two   that   intersect   in 
that  cell. 

2)  S   =  P  -  Q 

r-1  c-1  r  c^ 

where  P   ■  S         2Z       S  22  X     X 

isl  j=l  k=i+l         <=j+l  J      C 

r-1  c  r  j-1 

Q=2I  Tl      XI  Tl  XX 

i=l  j=2  ksifl         ZZl  LJ   KC 

3)  For  square   tables    (where  r=c),    Kendall's    tau-B  was 
computed  as    follows: 

tau-B   s  S 


V*N(N-1)-T    \AN(N-1)-U 
where  T  and  U  are  factors  representing  the  number 
of  ties  in  the  ranking  of  each  variable,  and  are 
computed  as  follows: 


T  =  1/2  51  n   (n   -1) 

i-i  l:   i* 

X^n.jCn.j-l 


U  s  1/2  2_.n.t(n.t-l) 

4)  For  rectangular  tables  (where  r^c),  Kendall's  tau-C 
was  computed  as  follows: 
tau-C  s     2S 


N  \  m  / 


where  m  is  the  smaller  of  r  or  c 
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5)  The  normal  deviate  of  S  can  be  computed  only  after 
the  variance  of  the  S  statistic  has  been  determined 
as  follows: 
VAR  S  s 


4- 


~  h(N-l)  (2Nf5)-^n   (nt  -l)(2n  +  5) 
L  isl1* 

-/,    n.  (n.  -l)(2n.  ,+5) 


9N(N-l)(N-2) 


Li=l 


S.n.    in.    -l)(n. .-2) 
J=l   J    J      J 


2N(N-1) 


"i.^i."1) 


Lin 


rn.  .(n.  .-1) 


3=1 


6)  The  normal  deviate  of  S  can  next  be  computed  by: 
Z(s)  «    S* 


Vvar  S 
*Before  computing  the  normal  deviate,  S  is 
corrected  for  continuity  as  follows: 

a)  for  a  2  x  2  table,  |sj  reduced  by  ^N 
unless  S  <  IjN,  in  which  case  S  s  0. 

b)  for  all  other  tables,   S   is  reduced  by  1. 
These  corrections  for  continuity  are  made  to 
adjust  the  distribution  of  S  closer  to  the  normal 
distribution  where  there  are  ties  in  either  or 
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both  of  the  rankings  being  cross-classified. 
However,  where  the  extent  of  tied  ranks  is 
very  great,  the  correction  in  (b)  above  may 
not  be  enough.   This  is  particularly  true 
where  one  ranking  is  a  dichotomy  and  the  other 
ranking  has  ties  of  varying  extents  in  it. 
In  these  last  cases,  an  actual  distribution  of 
S  for  each  set  of  variables  cross-classified 
would  need  to  be  determined  to  arrive  at  the 
continuity  correction  required. 


For  discussions  and  proofs  of  the  correction  for 
continuity  applied  to  S  under  various  cases  of  tied  ranks, 
see  Burr  (1960),  Kendall  (1947;  1962),  Sillitto  (1947),  and 
Whitfield  (1947). 
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APPENDIX  G 
COMMENTS  OF  USER  MANAGEMENT 

Each  user  interviewed  was  given  the  opportunity  at 
the  end  of  the  interview  to  make  whatever  comments  he  felt 
were  germane  to  the  project  being  studied.   Specifically,  each 
user  was  asked  what  he  felt  should  have  been  done  to  improve 
the  effectiveness  and  value  of  the  project  with  respect  to  his 
information  needs.   Further,  the  user  was  asked  to  elaborate 
on  any  aspects  of  the  project  which  he  felt  required  additional 
development  work. 

The  comments  included  in  this  appendix  provide  some 
flavor  of  the  attitudes  of  user  managers  toward  the  projects 
studied.   Since  verbatim  notes  were  not  taken  by  the  researcher, 
the  comments  presented  here  are  best  described  as  "paraphrased 
user  comments".   However,  the  comments  are  presented  in  the  form 
of  statements  by  the  users  rather  than  descriptive  narratives 
of  the  researcher. 

1.   Problems  that  were  apparent  from  the  beginning 
have  never  been  corrected.   There  has  been 
virtually  no  follow-through  by  the  project 
staff  to  help  us  work  with  the  system.  We 
have  also  been  unhappy  with  the  validity  of 
the  input  data;  we  don't  consider  the  reports 
we're  getting  to  be  very  reliable. 
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2.  We  have  learned  a  great  deal  from  the 
experience  of  working  with  the  system. 
We  now  feel  there  is  too  much  historical 
information  provided  in  the  outputs; 
completed  transactions  should  be  purged 
from  the  files  so  that  they  do  not  appear 
in  our  reports.   Also,  there  has  been  a 
lack  of  training  for  the  people  preparing 
the  input  data.   This  has  resulted  in  too 
many  input  errors  which  makes  us  suspicious 
of  the  validity  of  the  outputs. 

3.  There  was  too  much  pressure  from  top  manage- 
ment to  get  something  going  in  a  hurry  to 
meet  an  immediate  problem.   This  led  to 
inadequate  planning  and  preparation.   I 
think  we  are  trying  to  do  too  much  with  the 
system;  we  should  reduce  the  scope  of  this 
project  and  focus  just  on  the  heart  of  the 
problem,  rather  than  trying  to  do  everything 
for  everyone  with  one  project. 

4.  We  want  a  good  deal  more  now  than  we  envisioned 
needing  when  the  system  was  designed.   The 
original  outputs  were  fine,  based  on  what  we 
said  we  wanted,  but  we  have  been  unable  to 

get  changes  made  that  we  feel  are  necessary  now. 
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Another  problem  was  that  the  salesmen  who 
provide  the  input  were  never  involved  in 
system  design.   As  a  consequence,  they 
cannot  see  any  results  from  the  coding 
they  are  doing— they  see  no  specific  outcome 
and  tend  to  feel  they  are  wasting  their  time. 

5.  This  is  a  very  valuable  application.   The 
opportunity  to  evolve  with  it  and  to  incor- 
porate needed  changes  in  phase  two  was 
crucial  to  our  getting  useful  outputs. 

6.  We  are  just  getting  too  much  paper.   We 
need  reports  oriented  to  the  needs  of  the 
individual  decision  maker. 

7.  From  our  standpoint,  the  system  is  not 
worth  too  much.   The  production  engineers 
who  are  supposed  to  use  the  system  had 
little  or  no  voice  in  designing  it.   Higher 
level  managers  were  involved  in  defining 
the  system,  which  we  believe  resulted  in 
outputs  more  oriented  to  accounting  needs 
than  to  production  requirements.   The 
reports  are  too  voluminous  and  hard  to 

use;  there  should  be  more  exception  reporting. 

8.  I  get  a  lot  of  performance  figures,  but  these 
are  meaningless  to  me  without  some  standard 
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or  scale  for  comparison.     What  is  good,  bad, 
acceptable?     I  don't  believe   there   is   a 
real  means   for  a  manager   in  my  position  to 
use  the  system— to  really  be  an  effective 
influence  on  actions   taken.     Although  some 
of   the  other  managers  were   involved   in 
developing   the  system,    I  don't  believe   I 
had  enough  voice   in  it. 
9.     We  don't  get  the  output  in  a   format  that 
is  usable.     Our  analysts  still  have  to 
rearrange  the  data  and  prepare  the  required 
reports  by  hand.      However,      I  think   the 
blame  for   this  situation  falls  on  us— we 
did  not  state  explicitly  enough  what  was 
needed. 

10.  Eighty   to  ninety  percent  of  our  information 
requires  output  formatting  of  a  certain  type. 
We  don't  get  that  type  of  output  at  all.     The 
need  for   this   type  of  output  format  was  very 

clearly  specified   in  the  original  project 

1 
request. 

11.  It  seems  that  the  original  objectives  were 
satisfied  well  enough,  but  it  is  apparent  now 


Comments  9  and  10  came  from  two  different  users  of  the 
project  outputs  in  the  same  organization. 
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that  we  did  not  really  know  what  we  wanted 
or  needed  when  we  started.   We  have  grown 
with  experience  with  the  present  system--we  now 
have  a  better  grasp  of  what  information  we 
should  be  getting  but  are  unable  to  get  it 
with  the  present  system.   Among  the  specific 
shortcomings,  as  I  see  them,  are:   the 
information  is  not  now  current  or  accurate 
enough  to  make  the  kinds  of  decisions  we 
should  be  making;  reports  are  too  detailed 
and  voluminous --we  need  much  more  analytical 
reports  that  satisfy  our  information  needs 
directly  without  further  manipulation;  and 
too  many  people  are  involved  in  data  prepara- 
tion with  no  effective  input  quality  control. 

12.  The  form  required  to  be  filled  out  for  an 
inquiry  run  is  rather  complex.   As  a  result, 
some  potential  users  have  not  used  the  system 
as  they  could.   I  guess  this  may  partly  stem 
from  the  fact  that  all  potential  user  areas 
were  not  well  enough  represented  and  involved 
in  original  system  definition. 

13.  Due  to  the  way  the  project  was  developed,  and 
the  way  it  is  now  used,  the  field  sales  force 
is  very  opposed  to  the  forecasting  system. 
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They  were  not  consulted  In  the  development 
efforts,  and  they  have  no  Input  to  the  processing 
cycle  now.   However,  I  guess  there  has  been  a 
time  constraint  that  precluded  our  including 
the  field  sales  force  in  the  processing  cycle. 

14.  Very  simply,  we  did  not  ask  for  enough;  we 
did  not  specify  what  we  wanted  clearly  enough; 
and  the  result  is  that  we  are  still  doing  too 
much  manually. 

15.  I  can  think  of  a  couple  of  things  that  are 
probably  related.   First,  the  documentation  has 
been  inadequate  in  that  changes  to  programs 
have  often  not  been  reflected  in  documentation 
nor  communicated  to  users.   Second,  the  field 
training  of  operating  people  has  been  inadequate 
which,  I  think,  accounts  for  some  of  their  resist- 
ance to  the  system. 

16.  There  are  too  many  inaccuracies  in  the  data-- 
inadequate  input  controls  and  data  validation. 
Also,  the  reports  are  too  detailed  and  difficult 
to  use  for  vice  presidents  and  assistant  vice 
presidents.   Finally,  we  have  had  implementation 
problems.   We  are  not  getting  the  reports  where 
and  when  we  should  in  some  cases.   This,  combined 
with  inaccuracies,  is  frustrating. 
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